Jul 20 21:00:27.445048 ip-10-0-128-253 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 20 21:00:27.445065 ip-10-0-128-253 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 20 21:00:27.445074 ip-10-0-128-253 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 20 21:00:27.445621 ip-10-0-128-253 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 20 21:00:37.545675 ip-10-0-128-253 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 20 21:00:37.545696 ip-10-0-128-253 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 56479af0c7d94202b6bbcf34f21349a9 --
Jul 20 21:02:37.661222 ip-10-0-128-253 systemd[1]: Starting Kubernetes Kubelet...
Jul 20 21:02:38.138614 ip-10-0-128-253 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 21:02:38.138614 ip-10-0-128-253 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 20 21:02:38.138614 ip-10-0-128-253 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 21:02:38.138614 ip-10-0-128-253 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 20 21:02:38.138614 ip-10-0-128-253 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 21:02:38.138614 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.087889    2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 20 21:02:38.138614 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095245    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 21:02:38.138614 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095259    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 21:02:38.138614 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095263    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 21:02:38.138614 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095267    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 21:02:38.138614 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095270    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 21:02:38.138614 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095273    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 21:02:38.138614 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095275    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 21:02:38.138614 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095278    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 21:02:38.138614 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095280    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 21:02:38.140649 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095283    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 21:02:38.140649 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095286    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 21:02:38.140649 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095291    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 21:02:38.140649 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095296    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 21:02:38.140649 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095300    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 21:02:38.140649 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095303    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 21:02:38.140649 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095306    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 21:02:38.140649 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095308    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 21:02:38.140649 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095311    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 21:02:38.140649 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095314    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 21:02:38.140649 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095316    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 21:02:38.140649 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095319    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 21:02:38.140649 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095321    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 21:02:38.140649 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095324    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 21:02:38.140649 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095327    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 21:02:38.140649 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095330    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 21:02:38.140649 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095333    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 21:02:38.140649 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095335    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 21:02:38.140649 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095338    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 21:02:38.141302 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095341    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 21:02:38.141302 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095343    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 21:02:38.141302 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095346    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 21:02:38.141302 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095349    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 21:02:38.141302 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095353    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 21:02:38.141302 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095357    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 21:02:38.141302 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095360    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 21:02:38.141302 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095362    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 21:02:38.141302 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095365    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 21:02:38.141302 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095368    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 21:02:38.141302 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095370    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 21:02:38.141302 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095373    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 21:02:38.141302 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095376    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 21:02:38.141302 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095378    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 21:02:38.141302 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095381    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 21:02:38.141302 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095384    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 21:02:38.141302 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095387    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 21:02:38.141302 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095389    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 21:02:38.141302 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095392    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 21:02:38.142264 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095395    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 21:02:38.142264 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095397    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 21:02:38.142264 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095400    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 21:02:38.142264 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095402    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 21:02:38.142264 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095405    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 21:02:38.142264 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095407    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 21:02:38.142264 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095410    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 21:02:38.142264 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095412    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 21:02:38.142264 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095415    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 21:02:38.142264 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095419    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 21:02:38.142264 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095422    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 21:02:38.142264 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095424    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 21:02:38.142264 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095427    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 21:02:38.142264 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095430    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 21:02:38.142264 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095433    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 21:02:38.142264 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095436    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 21:02:38.142264 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095438    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 21:02:38.142264 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095441    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 21:02:38.142264 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095443    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 21:02:38.142264 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095446    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 21:02:38.143222 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095448    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 21:02:38.143222 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095451    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 21:02:38.143222 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095453    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 21:02:38.143222 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095456    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 21:02:38.143222 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095460    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 21:02:38.143222 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095463    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 21:02:38.143222 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095465    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 21:02:38.143222 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095468    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 21:02:38.143222 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095471    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 21:02:38.143222 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095473    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 21:02:38.143222 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095476    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 21:02:38.143222 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095478    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 21:02:38.143222 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095481    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 21:02:38.143222 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095483    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 21:02:38.143222 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095486    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 21:02:38.143222 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095488    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 21:02:38.143222 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095490    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 21:02:38.143222 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095493    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 21:02:38.143222 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095495    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 21:02:38.143222 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095498    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 21:02:38.144129 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095500    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 21:02:38.144129 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095503    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 21:02:38.144129 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095506    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 21:02:38.144129 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095508    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 21:02:38.144129 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095511    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 21:02:38.144129 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095514    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 21:02:38.144129 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095517    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 21:02:38.144129 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095519    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 21:02:38.144129 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095523    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 21:02:38.144129 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095527    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 21:02:38.144129 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095529    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 21:02:38.144129 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.095533    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 21:02:38.144129 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096353    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 21:02:38.144129 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096360    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 21:02:38.144129 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096363    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 21:02:38.144129 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096366    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 21:02:38.144129 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096369    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 21:02:38.144129 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096378    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 21:02:38.144129 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096382    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 21:02:38.144129 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096384    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 21:02:38.144859 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096387    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 21:02:38.144859 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096390    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 21:02:38.144859 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096393    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 21:02:38.144859 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096395    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 21:02:38.144859 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096398    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 21:02:38.144859 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096400    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 21:02:38.144859 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096403    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 21:02:38.144859 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096406    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 21:02:38.144859 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096408    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 21:02:38.144859 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096411    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 21:02:38.144859 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096413    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 21:02:38.144859 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096416    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 21:02:38.144859 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096420    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 21:02:38.144859 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096424    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 21:02:38.144859 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096427    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 21:02:38.144859 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096430    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 21:02:38.144859 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096435    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 21:02:38.144859 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096438    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 21:02:38.144859 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096441    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 21:02:38.145453 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096444    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 21:02:38.145453 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096446    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 21:02:38.145453 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096449    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 21:02:38.145453 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096452    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 21:02:38.145453 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096454    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 21:02:38.145453 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096456    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 21:02:38.145453 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096459    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 21:02:38.145453 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096461    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 21:02:38.145453 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096464    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 21:02:38.145453 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096466    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 21:02:38.145453 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096469    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 21:02:38.145453 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096474    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 21:02:38.145453 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096476    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 21:02:38.145453 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096479    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 21:02:38.145453 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096482    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 21:02:38.145453 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096484    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 21:02:38.145453 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096487    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 21:02:38.145453 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096489    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 21:02:38.145453 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096492    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 21:02:38.146209 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096495    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 21:02:38.146209 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096497    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 21:02:38.146209 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096500    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 21:02:38.146209 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096502    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 21:02:38.146209 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096505    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 21:02:38.146209 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096507    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 21:02:38.146209 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096510    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 21:02:38.146209 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096514    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 21:02:38.146209 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096517    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 21:02:38.146209 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096520    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 21:02:38.146209 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096523    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 21:02:38.146209 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096526    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 21:02:38.146209 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096528    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 21:02:38.146209 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096531    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 21:02:38.146209 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096533    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 21:02:38.146209 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096536    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 21:02:38.146209 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096539    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 21:02:38.146209 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096542    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 21:02:38.146209 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096546    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 21:02:38.146209 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096550    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 21:02:38.146781 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096553    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 21:02:38.146781 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096556    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 21:02:38.146781 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096559    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 21:02:38.146781 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096563    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 21:02:38.146781 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096566    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 21:02:38.146781 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096569    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 21:02:38.146781 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096572    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 21:02:38.146781 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096575    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 21:02:38.146781 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096577    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 21:02:38.146781 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096580    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 21:02:38.146781 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096582    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 21:02:38.146781 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096585    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 21:02:38.146781 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096588    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 21:02:38.146781 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096590    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 21:02:38.146781 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096593    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 21:02:38.146781 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096596    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 21:02:38.146781 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096598    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 21:02:38.146781 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096601    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 21:02:38.146781 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096604    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 21:02:38.146781 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096606    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 21:02:38.147340 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096609    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 21:02:38.147340 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096611    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 21:02:38.147340 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096615    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 21:02:38.147340 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096618    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 21:02:38.147340 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096621    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 21:02:38.147340 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096624    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 21:02:38.147340 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096627    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 21:02:38.147340 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096630    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 21:02:38.147340 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096633    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 21:02:38.147340 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096639    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 21:02:38.147340 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096641    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 21:02:38.147340 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096644    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 21:02:38.147340 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.096646    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 21:02:38.147340 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097066    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 21:02:38.147340 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097072    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 21:02:38.147340 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097075    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 21:02:38.147340 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097078    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 21:02:38.147340 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097082    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 21:02:38.147340 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097085    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 21:02:38.147901 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097087    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 21:02:38.147901 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097090    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 21:02:38.147901 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097093    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 21:02:38.147901 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097095    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 21:02:38.147901 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097098    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 21:02:38.147901 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097101    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 21:02:38.147901 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097103    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 21:02:38.147901 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097106    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 21:02:38.147901 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097110    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 21:02:38.147901 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097113    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 21:02:38.147901 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097116    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 21:02:38.147901 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097120    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 21:02:38.147901 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097122    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 21:02:38.147901 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097125    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 21:02:38.147901 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097128    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 21:02:38.147901 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097131    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 21:02:38.147901 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097134    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 21:02:38.147901 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097137    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 21:02:38.147901 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097140    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 21:02:38.148484 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097144    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 21:02:38.148484 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097146    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 21:02:38.148484 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097149    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 21:02:38.148484 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097154    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 21:02:38.148484 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097157    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 21:02:38.148484 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097160    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 21:02:38.148484 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097163    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 21:02:38.148484 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097167    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 21:02:38.148484 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097170    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 21:02:38.148484 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097172    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 21:02:38.148484 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097177    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 21:02:38.148484 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097180    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 21:02:38.148484 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097185    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 21:02:38.148484 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097188    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 21:02:38.148484 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097191    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 21:02:38.148484 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097194    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 21:02:38.148484 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097197    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 21:02:38.148484 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097200    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 21:02:38.148484 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097202    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 21:02:38.149044 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097205    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 21:02:38.149044 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097208    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 21:02:38.149044 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097210    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 21:02:38.149044 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097213    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 21:02:38.149044 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097216    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 21:02:38.149044 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097218    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 21:02:38.149044 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097221    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 21:02:38.149044 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097224    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 21:02:38.149044 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097226    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 21:02:38.149044 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097229    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 21:02:38.149044 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097232    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 21:02:38.149044 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097234    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 21:02:38.149044 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097237    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 21:02:38.149044 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097239    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 21:02:38.149044 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097242    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 21:02:38.149044 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097244    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 21:02:38.149044 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097249    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 21:02:38.149044 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097252    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 21:02:38.149044 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097254    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 21:02:38.149619 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097257    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 21:02:38.149619 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097259    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 21:02:38.149619 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097262    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 21:02:38.149619 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097265    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 21:02:38.149619 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097267    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 21:02:38.149619 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097270    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 21:02:38.149619 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097274    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 21:02:38.149619 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097280    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 21:02:38.149619 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097283    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 21:02:38.149619 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097286    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 21:02:38.149619 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097288    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 21:02:38.149619 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097290    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 21:02:38.149619 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097293    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 21:02:38.149619 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097295    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 21:02:38.149619 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097298    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 21:02:38.149619 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097300    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 21:02:38.149619 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097303    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 21:02:38.149619 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097305    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 21:02:38.149619 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097308    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 21:02:38.149619 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097310    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 21:02:38.150224 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097313    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 21:02:38.150224 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097316    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 21:02:38.150224 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097318    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 21:02:38.150224 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097320    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 21:02:38.150224 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097323    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 21:02:38.150224 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097326    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 21:02:38.150224 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097328    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 21:02:38.150224 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097331    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 21:02:38.150224 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097333    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 21:02:38.150224 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097337    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 21:02:38.150224 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097340    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 21:02:38.150224 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097342    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 21:02:38.150224 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097345    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 21:02:38.150224 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097347    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 21:02:38.150224 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097350    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 21:02:38.150224 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.097352    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 21:02:38.150224 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097419    2561 flags.go:64] FLAG: --address="0.0.0.0"
Jul 20 21:02:38.150224 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097431    2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 20 21:02:38.150224 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097441    2561 flags.go:64] FLAG: --anonymous-auth="true"
Jul 20 21:02:38.150224 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097448    2561 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 20 21:02:38.150224 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097456    2561 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 20 21:02:38.150224 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097461    2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 20 21:02:38.150977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097466    2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 20 21:02:38.150977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097470    2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 20 21:02:38.150977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097473    2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 20 21:02:38.150977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097476    2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 20 21:02:38.150977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097480    2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 20 21:02:38.150977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097483    2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 20 21:02:38.150977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097486    2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 20 21:02:38.150977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097489    2561 flags.go:64] FLAG: --cgroup-root=""
Jul 20 21:02:38.150977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097492    2561 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 20 21:02:38.150977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097495    2561 flags.go:64] FLAG: --client-ca-file=""
Jul 20 21:02:38.150977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097498    2561 flags.go:64] FLAG: --cloud-provider="external"
Jul 20 21:02:38.150977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097501    2561 flags.go:64] FLAG: --cluster-dns="[]"
Jul 20 21:02:38.150977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097506    2561 flags.go:64] FLAG: --cluster-domain=""
Jul 20 21:02:38.150977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097509    2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 20 21:02:38.150977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097512    2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 20 21:02:38.150977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097515    2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 20 21:02:38.150977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097518    2561 flags.go:64] FLAG: --container-log-max-files="5"
Jul 20 21:02:38.150977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097522    2561 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 20 21:02:38.150977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097525    2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 20 21:02:38.150977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097529    2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 20 21:02:38.150977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097533    2561 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 20 21:02:38.150977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097536    2561 flags.go:64] FLAG: --contention-profiling="false"
Jul 20 21:02:38.150977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097539    2561 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 20 21:02:38.150977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097542    2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 20 21:02:38.151831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097545    2561 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 20 21:02:38.151831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097548    2561 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 20 21:02:38.151831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097552    2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 20 21:02:38.151831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097555    2561 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 20 21:02:38.151831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097559    2561 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 20 21:02:38.151831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097562    2561 flags.go:64] FLAG: --enable-load-reader="false"
Jul 20 21:02:38.151831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097566    2561 flags.go:64] FLAG: --enable-server="true"
Jul 20 21:02:38.151831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097570    2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 20 21:02:38.151831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097574    2561 flags.go:64] FLAG: --event-burst="100"
Jul 20 21:02:38.151831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097577    2561 flags.go:64] FLAG: --event-qps="50"
Jul 20 21:02:38.151831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097580    2561 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 20 21:02:38.151831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097583    2561 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 20 21:02:38.151831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097586    2561 flags.go:64] FLAG: --eviction-hard=""
Jul 20 21:02:38.151831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097590    2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 20 21:02:38.151831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097593    2561 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 20 21:02:38.151831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097596    2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 20 21:02:38.151831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097599    2561 flags.go:64] FLAG: --eviction-soft=""
Jul 20 21:02:38.151831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097602    2561 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 20 21:02:38.151831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097605    2561 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 20 21:02:38.151831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097607    2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 20 21:02:38.151831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097610    2561 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 20 21:02:38.151831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097613    2561 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 20 21:02:38.151831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097616    2561 flags.go:64] FLAG: --fail-swap-on="true"
Jul 20 21:02:38.151831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097620    2561 flags.go:64] FLAG: --feature-gates=""
Jul 20 21:02:38.151831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097624    2561 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 20 21:02:39.578083 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097627    2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 20 21:02:39.578083 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097630    2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 20 21:02:39.578083 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097635    2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 20 21:02:39.578083 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097638    2561 flags.go:64] FLAG: --healthz-port="10248"
Jul 20 21:02:39.578083 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097641    2561 flags.go:64] FLAG: --help="false"
Jul 20 21:02:39.578083 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097644    2561 flags.go:64] FLAG: --hostname-override="ip-10-0-128-253.ec2.internal"
Jul 20 21:02:39.578083 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097647    2561 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 20 21:02:39.578083 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097651    2561 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 20 21:02:39.578083 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097653    2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 20 21:02:39.578083 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097657    2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 20 21:02:39.578083 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097660    2561 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 20 21:02:39.578083 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097664    2561 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 20 21:02:39.578083 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097668    2561 flags.go:64] FLAG: --image-service-endpoint=""
Jul 20 21:02:39.578083 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097670    2561 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 20 21:02:39.578083 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097673    2561 flags.go:64] FLAG: --kube-api-burst="100"
Jul 20 21:02:39.578083 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097676    2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 20 21:02:39.578083 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097680    2561 flags.go:64] FLAG: --kube-api-qps="50"
Jul 20 21:02:39.578083 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097703    2561 flags.go:64] FLAG: --kube-reserved=""
Jul 20 21:02:39.578083 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097707    2561 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 20 21:02:39.578083 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097710    2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 20 21:02:39.578083 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097714    2561 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 20 21:02:39.578083 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097717    2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 20 21:02:39.578083 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097720    2561 flags.go:64] FLAG: --lock-file=""
Jul 20 21:02:39.578083 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097723    2561 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 20 21:02:38.176542 ip-10-0-128-253 systemd[1]: Started Kubernetes Kubelet.
Jul 20 21:02:39.579369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097726    2561 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 20 21:02:39.579369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097729    2561 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 20 21:02:39.579369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097734    2561 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 20 21:02:39.579369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097737    2561 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 20 21:02:39.579369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097740    2561 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 20 21:02:39.579369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097742    2561 flags.go:64] FLAG: --logging-format="text"
Jul 20 21:02:39.579369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097745    2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 20 21:02:39.579369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097749    2561 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 20 21:02:39.579369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097752    2561 flags.go:64] FLAG: --manifest-url=""
Jul 20 21:02:39.579369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097755    2561 flags.go:64] FLAG: --manifest-url-header=""
Jul 20 21:02:39.579369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097761    2561 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 20 21:02:39.579369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097764    2561 flags.go:64] FLAG: --max-open-files="1000000"
Jul 20 21:02:39.579369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097768    2561 flags.go:64] FLAG: --max-pods="110"
Jul 20 21:02:39.579369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097771    2561 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 20 21:02:39.579369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097774    2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 20 21:02:39.579369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097777    2561 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 20 21:02:39.579369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097780    2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 20 21:02:39.579369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097783    2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 20 21:02:39.579369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097786    2561 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 20 21:02:39.579369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097790    2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 20 21:02:39.579369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097798    2561 flags.go:64] FLAG: --node-status-max-images="50"
Jul 20 21:02:39.579369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097801    2561 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 20 21:02:39.579369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097804    2561 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 20 21:02:39.579369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097808    2561 flags.go:64] FLAG: --pod-cidr=""
Jul 20 21:02:39.580354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097812    2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 20 21:02:39.580354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097818    2561 flags.go:64] FLAG: --pod-manifest-path=""
Jul 20 21:02:39.580354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097821    2561 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 20 21:02:39.580354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097824    2561 flags.go:64] FLAG: --pods-per-core="0"
Jul 20 21:02:39.580354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097827    2561 flags.go:64] FLAG: --port="10250"
Jul 20 21:02:39.580354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097830    2561 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 20 21:02:39.580354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097833    2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04d012a39d212aac8"
Jul 20 21:02:39.580354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097836    2561 flags.go:64] FLAG: --qos-reserved=""
Jul 20 21:02:39.580354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097839    2561 flags.go:64] FLAG: --read-only-port="10255"
Jul 20 21:02:39.580354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097842    2561 flags.go:64] FLAG: --register-node="true"
Jul 20 21:02:39.580354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097845    2561 flags.go:64] FLAG: --register-with-taints=""
Jul 20 21:02:39.580354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097849    2561 flags.go:64] FLAG: --registry-burst="10"
Jul 20 21:02:39.580354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097852    2561 flags.go:64] FLAG: --registry-qps="5"
Jul 20 21:02:39.580354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097854    2561 flags.go:64] FLAG: --reserved-cpus=""
Jul 20 21:02:39.580354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097857    2561 flags.go:64] FLAG: --reserved-memory=""
Jul 20 21:02:39.580354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097861    2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 20 21:02:39.580354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097864    2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 20 21:02:39.580354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097867    2561 flags.go:64] FLAG: --rotate-certificates="false"
Jul 20 21:02:39.580354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097872    2561 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 20 21:02:39.580354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097875    2561 flags.go:64] FLAG: --runonce="false"
Jul 20 21:02:39.580354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097878    2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 20 21:02:39.580354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097881    2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 20 21:02:39.580354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097884    2561 flags.go:64] FLAG: --seccomp-default="false"
Jul 20 21:02:39.580354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097887    2561 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 20 21:02:39.580354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097890    2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 20 21:02:39.581164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097893    2561 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 20 21:02:39.581164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097896    2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 20 21:02:39.581164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097899    2561 flags.go:64] FLAG: --storage-driver-password="root"
Jul 20 21:02:39.581164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097903    2561 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 20 21:02:39.581164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097906    2561 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 20 21:02:39.581164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097909    2561 flags.go:64] FLAG: --storage-driver-user="root"
Jul 20 21:02:39.581164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097912    2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 20 21:02:39.581164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097915    2561 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 20 21:02:39.581164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097919    2561 flags.go:64] FLAG: --system-cgroups=""
Jul 20 21:02:39.581164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097922    2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 20 21:02:39.581164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097943    2561 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 20 21:02:39.581164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097946    2561 flags.go:64] FLAG: --tls-cert-file=""
Jul 20 21:02:39.581164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097949    2561 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 20 21:02:39.581164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097954    2561 flags.go:64] FLAG: --tls-min-version=""
Jul 20 21:02:39.581164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097957    2561 flags.go:64] FLAG: --tls-private-key-file=""
Jul 20 21:02:39.581164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097960    2561 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 20 21:02:39.581164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097963    2561 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 20 21:02:39.581164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097966    2561 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 20 21:02:39.581164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097969    2561 flags.go:64] FLAG: --v="2"
Jul 20 21:02:39.581164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097972    2561 flags.go:64] FLAG: --version="false"
Jul 20 21:02:39.581164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097976    2561 flags.go:64] FLAG: --vmodule=""
Jul 20 21:02:39.581164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097980    2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 20 21:02:39.581164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.097984    2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 20 21:02:39.581164 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098121    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 21:02:39.581164 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098125    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 21:02:39.582024 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098130    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 21:02:39.582024 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098133    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 21:02:39.582024 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098136    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 21:02:39.582024 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098139    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 21:02:39.582024 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098142    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 21:02:39.582024 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098145    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 21:02:39.582024 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098147    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 21:02:39.582024 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098150    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 21:02:39.582024 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098153    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 21:02:39.582024 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098155    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 21:02:39.582024 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098159    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 21:02:39.582024 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098162    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 21:02:39.582024 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098164    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 21:02:39.582024 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098167    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 21:02:39.582024 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098170    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 21:02:39.582024 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098173    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 21:02:39.582024 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098181    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 21:02:39.582024 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098184    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 21:02:39.582024 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098187    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 21:02:39.582620 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098189    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 21:02:39.582620 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098192    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 21:02:39.582620 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098194    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 21:02:39.582620 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098196    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 21:02:39.582620 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098199    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 21:02:39.582620 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098202    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 21:02:39.582620 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098205    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 21:02:39.582620 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098208    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 21:02:39.582620 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098212    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 21:02:39.582620 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098214    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 21:02:39.582620 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098217    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 21:02:39.582620 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098219    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 21:02:39.582620 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098222    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 21:02:39.582620 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098226    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 21:02:39.582620 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098228    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 21:02:39.582620 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098231    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 21:02:39.582620 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098233    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 21:02:39.582620 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098236    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 21:02:39.582620 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098238    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 21:02:39.582620 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098241    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 21:02:39.583214 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098244    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 21:02:39.583214 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098247    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 21:02:39.583214 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098249    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 21:02:39.583214 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098254    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 21:02:39.583214 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098256    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 21:02:39.583214 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098259    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 21:02:39.583214 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098263    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 21:02:39.583214 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098266    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 21:02:39.583214 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098269    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 21:02:39.583214 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098272    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 21:02:39.583214 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098280    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 21:02:39.583214 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098283    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 21:02:39.583214 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098286    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 21:02:39.583214 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098288    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 21:02:39.583214 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098291    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 21:02:39.583214 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098294    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 21:02:39.583214 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098297    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 21:02:39.583214 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098299    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 21:02:39.583214 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098302    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 21:02:39.583838 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098305    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 21:02:39.583838 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098307    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 21:02:39.583838 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098310    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 21:02:39.583838 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098312    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 21:02:39.583838 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098315    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 21:02:39.583838 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098317    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 21:02:39.583838 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098321    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 21:02:39.583838 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098323    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 21:02:39.583838 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098326    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 21:02:39.583838 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098328    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 21:02:39.583838 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098331    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 21:02:39.583838 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098333    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 21:02:39.583838 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098336    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 21:02:39.583838 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098338    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 21:02:39.583838 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098341    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 21:02:39.583838 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098343    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 21:02:39.583838 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098347    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 21:02:39.583838 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098350    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 21:02:39.583838 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098352    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 21:02:39.583838 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098355    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 21:02:39.584476 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098358    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 21:02:39.584476 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098360    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 21:02:39.584476 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098363    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 21:02:39.584476 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098365    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 21:02:39.584476 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098369    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 21:02:39.584476 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098371    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 21:02:39.584476 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098374    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 21:02:39.584476 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098376    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 21:02:39.584476 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098378    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 21:02:39.584476 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098381    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 21:02:39.584476 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098384    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 21:02:39.584476 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098388    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 21:02:39.584476 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098391    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 21:02:39.584476 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098394    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 21:02:39.584476 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098396    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 21:02:39.584476 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098399    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 21:02:39.584476 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098408    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 21:02:39.584476 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098411    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 21:02:39.584476 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.098415    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 21:02:39.585110 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.099020    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 21:02:39.585110 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.113193    2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 20 21:02:39.585110 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.113212    2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 20 21:02:39.585110 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.113225    2561 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 20 21:02:39.585110 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.113231    2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 20 21:02:39.585110 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113295    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 21:02:39.585110 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113301    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 21:02:39.585110 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113305    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 21:02:39.585110 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113308    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 21:02:39.585110 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113311    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 21:02:39.585110 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113314    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 21:02:39.585110 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113317    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 21:02:39.585110 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113320    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 21:02:39.585110 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113323    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 21:02:39.585110 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113326    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 21:02:39.585642 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113329    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 21:02:39.585642 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113332    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 21:02:39.585642 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113336    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 21:02:39.585642 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113338    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 21:02:39.585642 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113341    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 21:02:39.585642 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113345    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 21:02:39.585642 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113349    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 21:02:39.585642 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113353    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 21:02:39.585642 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113358    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 21:02:39.585642 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113362    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 21:02:39.585642 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113364    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 21:02:39.585642 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113367    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 21:02:39.585642 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113371    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 21:02:39.585642 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113373    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 21:02:39.585642 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113377    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 21:02:39.585642 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113380    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 21:02:39.585642 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113383    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 21:02:39.585642 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113386    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 21:02:39.585642 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113389    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 21:02:39.585642 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113391    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 21:02:39.586285 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113394    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 21:02:39.586285 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113397    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 21:02:39.586285 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113399    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 21:02:39.586285 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113401    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 21:02:39.586285 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113404    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 21:02:39.586285 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113408    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 21:02:39.586285 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113412    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 21:02:39.586285 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113415    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 21:02:39.586285 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113418    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 21:02:39.586285 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113420    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 21:02:39.586285 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113424    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 21:02:39.586285 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113428    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 21:02:39.586285 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113432    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 21:02:39.586285 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113436    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 21:02:39.586285 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113442    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 21:02:39.586285 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113447    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 21:02:39.586285 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113450    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 21:02:39.586285 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113453    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 21:02:39.586285 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113456    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 21:02:39.586925 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113459    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 21:02:39.586925 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113461    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 21:02:39.586925 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113464    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 21:02:39.586925 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113468    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 21:02:39.586925 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113471    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 21:02:39.586925 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113474    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 21:02:39.586925 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113477    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 21:02:39.586925 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113483    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 21:02:39.586925 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113486    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 21:02:39.586925 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113488    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 21:02:39.586925 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113491    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 21:02:39.586925 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113493    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 21:02:39.586925 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113496    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 21:02:39.586925 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113498    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 21:02:39.586925 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113500    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 21:02:39.586925 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113503    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 21:02:39.586925 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113507    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 21:02:39.586925 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113511    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 21:02:39.586925 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113515    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 21:02:39.587819 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113519    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 21:02:39.587819 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113523    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 21:02:39.587819 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113526    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 21:02:39.587819 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113529    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 21:02:39.587819 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113532    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 21:02:39.587819 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113534    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 21:02:39.587819 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113537    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 21:02:39.587819 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113539    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 21:02:39.587819 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113542    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 21:02:39.587819 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113545    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 21:02:39.587819 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113549    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 21:02:39.587819 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113551    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 21:02:39.587819 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113555    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 21:02:39.587819 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113558    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 21:02:39.587819 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113560    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 21:02:39.587819 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113563    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 21:02:39.587819 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113566    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 21:02:39.587819 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113568    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 21:02:39.587819 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113571    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 21:02:39.587819 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113573    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 21:02:39.589106 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113577    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 21:02:39.589106 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113580    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 21:02:39.589106 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113583    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 21:02:39.589106 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113585    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 21:02:39.589106 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113589    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 21:02:39.589106 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113593    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 21:02:39.589106 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113597    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 21:02:39.589106 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113601    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 21:02:39.589106 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113605    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 21:02:39.589106 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113608    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 21:02:39.589106 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113610    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 21:02:39.589106 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.113615    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 21:02:39.589106 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113737    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 21:02:39.589106 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113742    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 21:02:39.589106 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113745    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 21:02:39.589930 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113748    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 21:02:39.589930 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113753    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 21:02:39.589930 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113757    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 21:02:39.589930 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113762    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 21:02:39.589930 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113767    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 21:02:39.589930 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113770    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 21:02:39.589930 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113774    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 21:02:39.589930 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113778    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 21:02:39.589930 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113786    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 21:02:39.589930 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113791    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 21:02:39.589930 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113793    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 21:02:39.589930 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113796    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 21:02:39.589930 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113799    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 21:02:39.589930 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113801    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 21:02:39.589930 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113804    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 21:02:39.589930 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113807    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 21:02:39.589930 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113809    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 21:02:39.589930 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113813    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 21:02:39.589930 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113816    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 21:02:39.589930 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113818    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 21:02:39.591537 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113821    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 21:02:39.591537 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113823    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 21:02:39.591537 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113826    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 21:02:39.591537 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113828    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 21:02:39.591537 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113832    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 21:02:39.591537 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113836    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 21:02:39.591537 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113840    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 21:02:39.591537 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113845    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 21:02:39.591537 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113849    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 21:02:39.591537 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113852    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 21:02:39.591537 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113854    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 21:02:39.591537 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113857    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 21:02:39.591537 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113861    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 21:02:39.591537 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113864    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 21:02:39.591537 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113867    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 21:02:39.591537 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113870    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 21:02:39.591537 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113872    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 21:02:39.591537 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113875    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 21:02:39.591537 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113878    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 21:02:39.592352 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113880    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 21:02:39.592352 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113883    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 21:02:39.592352 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113887    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 21:02:39.592352 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113890    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 21:02:39.592352 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113892    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 21:02:39.592352 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113895    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 21:02:39.592352 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113898    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 21:02:39.592352 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113900    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 21:02:39.592352 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113903    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 21:02:39.592352 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113905    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 21:02:39.592352 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113909    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 21:02:39.592352 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113915    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 21:02:39.592352 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113920    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 21:02:39.592352 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113938    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 21:02:39.592352 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113943    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 21:02:39.592352 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113958    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 21:02:39.592352 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113960    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 21:02:39.592352 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113963    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 21:02:39.592352 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113966    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 21:02:39.593387 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113968    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 21:02:39.593387 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113971    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 21:02:39.593387 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113973    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 21:02:39.593387 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113976    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 21:02:39.593387 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113979    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 21:02:39.593387 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113981    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 21:02:39.593387 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113984    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 21:02:39.593387 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113987    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 21:02:39.593387 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113989    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 21:02:39.593387 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113993    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 21:02:39.593387 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.113997    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 21:02:39.593387 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114001    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 21:02:39.593387 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114005    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 21:02:39.593387 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114010    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 21:02:39.593387 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114013    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 21:02:39.593387 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114017    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 21:02:39.593387 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114021    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 21:02:39.593387 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114024    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 21:02:39.593387 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114026    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 21:02:39.593387 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114029    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 21:02:39.593998 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114031    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 21:02:39.593998 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114034    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 21:02:39.593998 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114037    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 21:02:39.593998 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114039    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 21:02:39.593998 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114043    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 21:02:39.593998 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114045    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 21:02:39.593998 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114048    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 21:02:39.593998 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114051    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 21:02:39.593998 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114053    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 21:02:39.593998 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114057    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 21:02:39.593998 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114059    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 21:02:39.593998 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114062    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 21:02:39.593998 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114064    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 21:02:39.593998 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114067    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 21:02:39.593998 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114069    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 21:02:39.593998 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114072    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 21:02:39.593998 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114075    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 21:02:39.593998 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:38.114080    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 21:02:39.594609 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.114088    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 21:02:39.594609 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.114220    2561 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 20 21:02:39.594609 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.116558    2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 20 21:02:39.594609 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.117884    2561 server.go:1019] "Starting client certificate rotation"
Jul 20 21:02:39.594609 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.117961    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 20 21:02:39.594609 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.117999    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 20 21:02:39.594609 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.120096    2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 20 21:02:39.594609 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.123107    2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 20 21:02:39.594609 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.133987    2561 log.go:25] "Validated CRI v1 runtime API"
Jul 20 21:02:39.594609 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.136044    2561 log.go:25] "Validated CRI v1 image API"
Jul 20 21:02:39.594609 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.144624    2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 20 21:02:39.594609 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.149349    2561 fs.go:135] Filesystem UUIDs: map[786965bf-357b-43fc-9e25-4ed51485780c:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 a5dbc2fb-2c50-420c-b8f3-a38e7021aeb6:/dev/nvme0n1p3]
Jul 20 21:02:39.595000 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.149364    2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 20 21:02:39.595088 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.155671    2561 manager.go:216] Machine: {Timestamp:2026-07-20 21:02:38.153978478 +0000 UTC m=+0.372112703 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099608 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21ac9d7cbb54dcf13034756d543e8c SystemUUID:ec21ac9d-7cbb-54dc-f130-34756d543e8c BootID:56479af0-c7d9-4202-b6bb-cf34f21349a9 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:eb:b7:a9:1f:63 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:eb:b7:a9:1f:63 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fa:0f:b3:e9:cb:83 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 20 21:02:39.595088 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.155790    2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 20 21:02:39.595088 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.155882    2561 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 20 21:02:39.595088 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.158860    2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 20 21:02:39.595088 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.158888    2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-253.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 20 21:02:39.595088 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.159586    2561 topology_manager.go:138] "Creating topology manager with none policy"
Jul 20 21:02:39.595088 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.159597    2561 container_manager_linux.go:309] "Creating device plugin manager"
Jul 20 21:02:39.595088 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.159604    2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.160484    2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.161145    2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.161691    2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.164608    2561 state_mem.go:36] "Initialized new in-memory state store"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.164747    2561 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.164805    2561 kubelet.go:486] "Attempting to sync node with API server"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.164818    2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.164837    2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.164851    2561 kubelet.go:398] "Adding apiserver pod source"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.164863    2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.165295    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.165315    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.167734    2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.170042    2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.170076    2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.171105    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.171125    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.171131    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.171138    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.171148    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.171154    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.171163    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.171168    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.171177    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.171185    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.171201    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.171209    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.172207    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.172216    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.175569    2561 server.go:1268] "Started kubelet"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.175682    2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.175762    2561 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.175920    2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.177859    2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-253.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:38.178004    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-253.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.177868    2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:38.182318    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.183461    2561 server.go:310] "Adding debug handlers to kubelet server"
Jul 20 21:02:39.595396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.184480    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gw2k7"
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:38.185705    2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:38.185441    2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-253.ec2.internal.18c41b4a46d5f123  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-253.ec2.internal,UID:ip-10-0-128-253.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-253.ec2.internal,},FirstTimestamp:2026-07-20 21:02:38.175514915 +0000 UTC m=+0.393649141,LastTimestamp:2026-07-20 21:02:38.175514915 +0000 UTC m=+0.393649141,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-253.ec2.internal,}"
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.189695    2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.189706    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:38.189753    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-253.ec2.internal\" not found"
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.190217    2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.190244    2561 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.190263    2561 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.190516    2561 reconstruct.go:97] "Volume reconstruction finished"
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.190527    2561 reconciler.go:29] "Reconciler: start to sync state"
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.191495    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gw2k7"
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.192191    2561 factory.go:55] Registering systemd factory
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.192229    2561 factory.go:223] Registration of the systemd container factory successfully
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.192762    2561 factory.go:153] Registering CRI-O factory
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.192776    2561 factory.go:223] Registration of the crio container factory successfully
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.192826    2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.192859    2561 factory.go:103] Registering Raw factory
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.192886    2561 manager.go:1196] Started watching for new ooms in manager
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:38.193236    2561 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-253.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:38.193253    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:38.193350    2561 manager.go:349] Starting recovery of all containers
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.524619    2561 manager.go:354] Recovery completed
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.529063    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.530875    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.530904    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.530915    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientPID"
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.531441    2561 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.531450    2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.531460    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.531465    2561 state_mem.go:36] "Initialized new in-memory state store"
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.532755    2561 policy_none.go:49] "None policy: Start"
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.532768    2561 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.532779    2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 20 21:02:39.597778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.533851    2561 policy_none.go:47] "Start"
Jul 20 21:02:39.615721 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.534127    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 21:02:39.615721 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.534151    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 21:02:39.615721 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.534160    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientPID"
Jul 20 21:02:39.615721 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.587820    2561 manager.go:337] "Starting Device Plugin manager"
Jul 20 21:02:39.615721 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:39.588037    2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 20 21:02:39.615721 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.588052    2561 server.go:87] "Starting device plugin registration server"
Jul 20 21:02:39.615721 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.588361    2561 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 20 21:02:39.615721 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.588372    2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 20 21:02:39.615721 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.588470    2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 20 21:02:39.615721 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.588562    2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 20 21:02:39.615721 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.588574    2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 20 21:02:39.615721 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:39.589168    2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 20 21:02:39.615721 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:39.589214    2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-253.ec2.internal\" not found"
Jul 20 21:02:39.728948 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.688497    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 21:02:39.728948 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.689567    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 21:02:39.728948 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.689597    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 21:02:39.728948 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.689607    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientPID"
Jul 20 21:02:39.728948 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.689640    2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-253.ec2.internal"
Jul 20 21:02:39.728948 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.696010    2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-253.ec2.internal"
Jul 20 21:02:39.728948 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:39.696032    2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-253.ec2.internal\": node \"ip-10-0-128-253.ec2.internal\" not found"
Jul 20 21:02:39.728948 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:39.709613    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-253.ec2.internal\" not found"
Jul 20 21:02:39.728948 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.718301    2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 20 21:02:39.728948 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.719659    2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 20 21:02:39.728948 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.719698    2561 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 20 21:02:39.728948 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.719722    2561 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 20 21:02:39.728948 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:39.719759    2561 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 20 21:02:39.728948 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.722167    2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 21:02:39.894135 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:39.809986    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-253.ec2.internal\" not found"
Jul 20 21:02:39.894135 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.820124    2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-253.ec2.internal"]
Jul 20 21:02:39.894135 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.820197    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 21:02:39.894135 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.821830    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 21:02:39.894135 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.821860    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 21:02:39.894135 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.821870    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientPID"
Jul 20 21:02:39.894135 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.828040    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 21:02:39.894135 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.828142    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal"
Jul 20 21:02:39.894135 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.828170    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 21:02:39.894135 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.828797    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 21:02:39.894135 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.828824    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 21:02:39.894135 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.828832    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 21:02:39.894135 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.828838    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientPID"
Jul 20 21:02:39.894135 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.828851    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 21:02:39.894135 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.828861    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientPID"
Jul 20 21:02:39.894135 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.830106    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-253.ec2.internal"
Jul 20 21:02:39.894135 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.830128    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 21:02:39.894135 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.830783    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 21:02:39.894135 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.830806    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 21:02:39.894135 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.830816    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeHasSufficientPID"
Jul 20 21:02:39.894135 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:39.846688    2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-253.ec2.internal\" not found" node="ip-10-0-128-253.ec2.internal"
Jul 20 21:02:39.894135 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:39.854993    2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-253.ec2.internal\" not found" node="ip-10-0-128-253.ec2.internal"
Jul 20 21:02:40.041969 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.906562    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4cd988cd2c2cd61004aa8c00d8ce4a16-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal\" (UID: \"4cd988cd2c2cd61004aa8c00d8ce4a16\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal"
Jul 20 21:02:40.041969 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.906590    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/daa60d7c62b18ac55a74c9a2a524cd8f-config\") pod \"kube-apiserver-proxy-ip-10-0-128-253.ec2.internal\" (UID: \"daa60d7c62b18ac55a74c9a2a524cd8f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-253.ec2.internal"
Jul 20 21:02:40.041969 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:39.906613    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4cd988cd2c2cd61004aa8c00d8ce4a16-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal\" (UID: \"4cd988cd2c2cd61004aa8c00d8ce4a16\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal"
Jul 20 21:02:40.041969 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:39.910553    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-253.ec2.internal\" not found"
Jul 20 21:02:40.041969 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:40.007555    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4cd988cd2c2cd61004aa8c00d8ce4a16-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal\" (UID: \"4cd988cd2c2cd61004aa8c00d8ce4a16\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal"
Jul 20 21:02:40.041969 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:40.007594    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4cd988cd2c2cd61004aa8c00d8ce4a16-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal\" (UID: \"4cd988cd2c2cd61004aa8c00d8ce4a16\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal"
Jul 20 21:02:40.041969 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:40.007610    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/daa60d7c62b18ac55a74c9a2a524cd8f-config\") pod \"kube-apiserver-proxy-ip-10-0-128-253.ec2.internal\" (UID: \"daa60d7c62b18ac55a74c9a2a524cd8f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-253.ec2.internal"
Jul 20 21:02:40.041969 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:40.007661    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/daa60d7c62b18ac55a74c9a2a524cd8f-config\") pod \"kube-apiserver-proxy-ip-10-0-128-253.ec2.internal\" (UID: \"daa60d7c62b18ac55a74c9a2a524cd8f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-253.ec2.internal"
Jul 20 21:02:40.041969 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:40.007686    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4cd988cd2c2cd61004aa8c00d8ce4a16-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal\" (UID: \"4cd988cd2c2cd61004aa8c00d8ce4a16\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal"
Jul 20 21:02:40.041969 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:40.007683    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4cd988cd2c2cd61004aa8c00d8ce4a16-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal\" (UID: \"4cd988cd2c2cd61004aa8c00d8ce4a16\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal"
Jul 20 21:02:40.041969 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:40.011614    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-253.ec2.internal\" not found"
Jul 20 21:02:40.244030 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:40.112450    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-253.ec2.internal\" not found"
Jul 20 21:02:40.244030 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:40.149571    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal"
Jul 20 21:02:40.244030 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:40.157945    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-253.ec2.internal"
Jul 20 21:02:40.244030 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:40.212633    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-253.ec2.internal\" not found"
Jul 20 21:02:40.346219 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:40.313342    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-253.ec2.internal\" not found"
Jul 20 21:02:40.413862 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:40.413830    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-253.ec2.internal\" not found"
Jul 20 21:02:40.432014 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:40.431984    2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 20 21:02:40.432183 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:40.432164    2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 20 21:02:40.510177 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:40.510090    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-19 20:57:39 +0000 UTC" deadline="2028-04-29 15:20:49.65053127 +0000 UTC"
Jul 20 21:02:40.510177 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:40.510125    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15570h18m9.140411035s"
Jul 20 21:02:40.514189 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:40.514165    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-253.ec2.internal\" not found"
Jul 20 21:02:40.549109 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:40.549067    2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 21:02:40.604212 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:40.604181    2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-253.ec2.internal"
Jul 20 21:02:40.613005 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:40.612983    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 20 21:02:40.614900 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:40.614883    2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal"
Jul 20 21:02:40.626042 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:40.626020    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 20 21:02:40.742196 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:40.742165    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaa60d7c62b18ac55a74c9a2a524cd8f.slice/crio-df22ee010e0df6c85853f2e35e9d7d1911bec47e06c48c941754150ea768ae7b WatchSource:0}: Error finding container df22ee010e0df6c85853f2e35e9d7d1911bec47e06c48c941754150ea768ae7b: Status 404 returned error can't find the container with id df22ee010e0df6c85853f2e35e9d7d1911bec47e06c48c941754150ea768ae7b
Jul 20 21:02:40.742752 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:40.742721    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cd988cd2c2cd61004aa8c00d8ce4a16.slice/crio-2afc0bc1e2067a2d135f5c2ec43de60226c46c750e5684378bb9b9e630a87749 WatchSource:0}: Error finding container 2afc0bc1e2067a2d135f5c2ec43de60226c46c750e5684378bb9b9e630a87749: Status 404 returned error can't find the container with id 2afc0bc1e2067a2d135f5c2ec43de60226c46c750e5684378bb9b9e630a87749
Jul 20 21:02:40.743860 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:40.743845    2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 21:02:40.747000 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:40.746986    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 21:02:41.073574 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.073550    2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 21:02:41.479832 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.479754    2561 apiserver.go:52] "Watching apiserver"
Jul 20 21:02:41.491031 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.490991    2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 20 21:02:41.494419 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.494392    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf","openshift-dns/node-resolver-7k6k7","openshift-image-registry/node-ca-8zfld","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal","openshift-multus/network-metrics-daemon-khqfb","openshift-network-operator/iptables-alerter-wt9j5","kube-system/konnectivity-agent-bpvws","kube-system/kube-apiserver-proxy-ip-10-0-128-253.ec2.internal","openshift-cluster-node-tuning-operator/tuned-fhwv7","openshift-multus/multus-4hzwh","openshift-multus/multus-additional-cni-plugins-rzsh6","openshift-network-diagnostics/network-check-target-mr5nf","openshift-ovn-kubernetes/ovnkube-node-w6bqv","kube-system/global-pull-secret-syncer-g6ctc"]
Jul 20 21:02:41.496707 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.496677    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.499042 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.499021    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.500002 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.499977    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 20 21:02:41.500002 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.499994    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 20 21:02:41.500191 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.499975    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 20 21:02:41.500191 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.500163    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-2jzwn\""
Jul 20 21:02:41.500308 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.499977    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 20 21:02:41.501871 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.500930    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8zfld"
Jul 20 21:02:41.501871 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.501723    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 20 21:02:41.502988 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.502968    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 20 21:02:41.503213 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.503192    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 20 21:02:41.503291 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.503223    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 20 21:02:41.503485 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.503465    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 20 21:02:41.503705 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.503689    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bcnwv\""
Jul 20 21:02:41.503780 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.503710    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 20 21:02:41.503977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.503961    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wxbll\""
Jul 20 21:02:41.504107 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.504093    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 20 21:02:41.504165 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.504134    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 20 21:02:41.504384 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.504320    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 20 21:02:41.504384 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.504374    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:02:41.504507 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.504394    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 20 21:02:41.504507 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:41.504476    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6ctc" podUID="6c684495-6967-47c7-ba35-0b9b37110406"
Jul 20 21:02:41.505541 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.505521    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:02:41.505630 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:41.505594    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khqfb" podUID="3f6a3926-e94e-4981-9952-3ec7307a1e48"
Jul 20 21:02:41.509991 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.509932    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wt9j5"
Jul 20 21:02:41.512090 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.512054    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.513889 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.513431    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bpvws"
Jul 20 21:02:41.515295 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.514826    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7k6k7"
Jul 20 21:02:41.516181 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.516033    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a1111bd9-7008-4387-8e06-83019291010b-cni-binary-copy\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.519255 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.516064    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78npv\" (UniqueName: \"kubernetes.io/projected/0a812cf7-8e6f-48e2-a22a-29539fa392da-kube-api-access-78npv\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.519255 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.516886    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/58fafef1-89fd-4094-9993-70ab07fecd85-serviceca\") pod \"node-ca-8zfld\" (UID: \"58fafef1-89fd-4094-9993-70ab07fecd85\") " pod="openshift-image-registry/node-ca-8zfld"
Jul 20 21:02:41.519255 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.516906    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 20 21:02:41.519255 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.516914    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-multus-socket-dir-parent\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.519255 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.516939    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-host-run-netns\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.519255 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.516947    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 20 21:02:41.519255 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.516965    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-etc-kubernetes\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.519255 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.517006    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsv88\" (UniqueName: \"kubernetes.io/projected/a1111bd9-7008-4387-8e06-83019291010b-kube-api-access-hsv88\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.519255 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.517031    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-var-lib-openvswitch\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.519255 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.517056    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-run-openvswitch\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.519255 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.517093    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs\") pod \"network-metrics-daemon-khqfb\" (UID: \"3f6a3926-e94e-4981-9952-3ec7307a1e48\") " pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:02:41.519255 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.517123    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 20 21:02:41.519255 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.517243    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 20 21:02:41.519255 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.517314    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mh44d\""
Jul 20 21:02:41.519255 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.517124    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-run-systemd\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.519255 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.517434    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0a812cf7-8e6f-48e2-a22a-29539fa392da-env-overrides\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.520059 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.517477    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0a812cf7-8e6f-48e2-a22a-29539fa392da-ovn-node-metrics-cert\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.520059 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.517502    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n62fq\" (UniqueName: \"kubernetes.io/projected/58fafef1-89fd-4094-9993-70ab07fecd85-kube-api-access-n62fq\") pod \"node-ca-8zfld\" (UID: \"58fafef1-89fd-4094-9993-70ab07fecd85\") " pod="openshift-image-registry/node-ca-8zfld"
Jul 20 21:02:41.520059 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.517529    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6c684495-6967-47c7-ba35-0b9b37110406-kubelet-config\") pod \"global-pull-secret-syncer-g6ctc\" (UID: \"6c684495-6967-47c7-ba35-0b9b37110406\") " pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:02:41.520059 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.517575    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-multus-conf-dir\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.520059 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.517600    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0a812cf7-8e6f-48e2-a22a-29539fa392da-ovnkube-config\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.520059 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.517803    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 20 21:02:41.520059 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.517810    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 20 21:02:41.520059 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.517828    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-q44wf\""
Jul 20 21:02:41.520059 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.517852    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf"
Jul 20 21:02:41.520059 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.518101    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 20 21:02:41.520059 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.518504    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-z22ms\""
Jul 20 21:02:41.520059 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.518669    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 20 21:02:41.520059 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.518792    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-55fdk\""
Jul 20 21:02:41.520453 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.520421    2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 20 21:02:41.521198 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.520632    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 20 21:02:41.521198 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521026    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 20 21:02:41.521198 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521053    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 20 21:02:41.521198 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521059    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:41.521443 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521275    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wkfpt\""
Jul 20 21:02:41.521443 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521313    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0a812cf7-8e6f-48e2-a22a-29539fa392da-ovnkube-script-lib\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.521443 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521341    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/58fafef1-89fd-4094-9993-70ab07fecd85-host\") pod \"node-ca-8zfld\" (UID: \"58fafef1-89fd-4094-9993-70ab07fecd85\") " pod="openshift-image-registry/node-ca-8zfld"
Jul 20 21:02:41.521443 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521364    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c684495-6967-47c7-ba35-0b9b37110406-original-pull-secret\") pod \"global-pull-secret-syncer-g6ctc\" (UID: \"6c684495-6967-47c7-ba35-0b9b37110406\") " pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:02:41.521443 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521397    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd9mx\" (UniqueName: \"kubernetes.io/projected/3f6a3926-e94e-4981-9952-3ec7307a1e48-kube-api-access-jd9mx\") pod \"network-metrics-daemon-khqfb\" (UID: \"3f6a3926-e94e-4981-9952-3ec7307a1e48\") " pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:02:41.521443 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521420    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-host-var-lib-kubelet\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.521443 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521441    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-host-kubelet\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.521772 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521464    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-host-slash\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.521772 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521486    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-etc-openvswitch\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.521772 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521508    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-host-cni-netd\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.521772 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521534    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-system-cni-dir\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.521772 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521564    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-run-ovn\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.521772 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521626    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-node-log\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.521772 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521446    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 20 21:02:41.521772 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521653    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-host-run-ovn-kubernetes\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.521772 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521677    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-host-cni-bin\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.521772 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521725    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6c684495-6967-47c7-ba35-0b9b37110406-dbus\") pod \"global-pull-secret-syncer-g6ctc\" (UID: \"6c684495-6967-47c7-ba35-0b9b37110406\") " pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:02:41.521772 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521753    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-multus-cni-dir\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.522296 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521788    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-host-run-k8s-cni-cncf-io\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.522296 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521812    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a1111bd9-7008-4387-8e06-83019291010b-multus-daemon-config\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.522296 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521849    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-host-run-multus-certs\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.522296 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521871    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-log-socket\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.522296 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521900    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-cnibin\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.522296 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521929    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-os-release\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.522296 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521934    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:02:41.522296 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.521996    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-host-var-lib-cni-bin\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.522296 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:41.522000    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr5nf" podUID="7f5aea7c-8d2c-4073-af50-ac30377c7e98"
Jul 20 21:02:41.522296 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.522029    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-host-var-lib-cni-multus\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.522296 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.522054    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-hostroot\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.522296 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.522141    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-systemd-units\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.522296 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.522193    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-host-run-netns\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.522296 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.522230    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.531619 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.531530    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 20 21:02:41.533233 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.532552    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-f45bk\""
Jul 20 21:02:41.533233 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.532735    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 20 21:02:41.541040 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.540717    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-sqt64"
Jul 20 21:02:41.546652 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.546622    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-sqt64"
Jul 20 21:02:41.606018 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.605991    2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 20 21:02:41.623268 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623237    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-systemd-units\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.623403 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623279    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.623403 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623307    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-etc-kubernetes\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.623403 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623331    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/58fafef1-89fd-4094-9993-70ab07fecd85-serviceca\") pod \"node-ca-8zfld\" (UID: \"58fafef1-89fd-4094-9993-70ab07fecd85\") " pod="openshift-image-registry/node-ca-8zfld"
Jul 20 21:02:41.623403 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623355    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcabbaa9-9e52-49f0-8786-ae54a37210e3-tmp-dir\") pod \"node-resolver-7k6k7\" (UID: \"bcabbaa9-9e52-49f0-8786-ae54a37210e3\") " pod="openshift-dns/node-resolver-7k6k7"
Jul 20 21:02:41.623403 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623359    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-systemd-units\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.623403 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623376    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whxxn\" (UniqueName: \"kubernetes.io/projected/31d116a1-5993-427a-b6e2-d500f2a38c00-kube-api-access-whxxn\") pod \"iptables-alerter-wt9j5\" (UID: \"31d116a1-5993-427a-b6e2-d500f2a38c00\") " pod="openshift-network-operator/iptables-alerter-wt9j5"
Jul 20 21:02:41.623403 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623359    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.623713 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623410    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c11a809e-92e3-4887-9ca9-c1c3617c3c6c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8k9vf\" (UID: \"c11a809e-92e3-4887-9ca9-c1c3617c3c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf"
Jul 20 21:02:41.623713 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623532    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c11a809e-92e3-4887-9ca9-c1c3617c3c6c-device-dir\") pod \"aws-ebs-csi-driver-node-8k9vf\" (UID: \"c11a809e-92e3-4887-9ca9-c1c3617c3c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf"
Jul 20 21:02:41.623713 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623569    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1edc053e-71df-4f28-9552-0e7165b736d4-cnibin\") pod \"multus-additional-cni-plugins-rzsh6\" (UID: \"1edc053e-71df-4f28-9552-0e7165b736d4\") " pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:41.623713 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623593    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1edc053e-71df-4f28-9552-0e7165b736d4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzsh6\" (UID: \"1edc053e-71df-4f28-9552-0e7165b736d4\") " pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:41.623713 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623619    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9h54\" (UniqueName: \"kubernetes.io/projected/1edc053e-71df-4f28-9552-0e7165b736d4-kube-api-access-m9h54\") pod \"multus-additional-cni-plugins-rzsh6\" (UID: \"1edc053e-71df-4f28-9552-0e7165b736d4\") " pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:41.623713 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623647    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-etc-kubernetes\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.623713 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623681    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-var-lib-openvswitch\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.623713 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623692    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-etc-kubernetes\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.624160 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623722    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-run-openvswitch\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.624160 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623734    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-var-lib-openvswitch\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.624160 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623747    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs\") pod \"network-metrics-daemon-khqfb\" (UID: \"3f6a3926-e94e-4981-9952-3ec7307a1e48\") " pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:02:41.624160 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623775    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/31d116a1-5993-427a-b6e2-d500f2a38c00-host-slash\") pod \"iptables-alerter-wt9j5\" (UID: \"31d116a1-5993-427a-b6e2-d500f2a38c00\") " pod="openshift-network-operator/iptables-alerter-wt9j5"
Jul 20 21:02:41.624160 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623782    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-run-openvswitch\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.624160 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623800    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c11a809e-92e3-4887-9ca9-c1c3617c3c6c-etc-selinux\") pod \"aws-ebs-csi-driver-node-8k9vf\" (UID: \"c11a809e-92e3-4887-9ca9-c1c3617c3c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf"
Jul 20 21:02:41.624160 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623825    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-run-systemd\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.624160 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623838    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/58fafef1-89fd-4094-9993-70ab07fecd85-serviceca\") pod \"node-ca-8zfld\" (UID: \"58fafef1-89fd-4094-9993-70ab07fecd85\") " pod="openshift-image-registry/node-ca-8zfld"
Jul 20 21:02:41.624160 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623849    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n62fq\" (UniqueName: \"kubernetes.io/projected/58fafef1-89fd-4094-9993-70ab07fecd85-kube-api-access-n62fq\") pod \"node-ca-8zfld\" (UID: \"58fafef1-89fd-4094-9993-70ab07fecd85\") " pod="openshift-image-registry/node-ca-8zfld"
Jul 20 21:02:41.624160 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623872    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6c684495-6967-47c7-ba35-0b9b37110406-kubelet-config\") pod \"global-pull-secret-syncer-g6ctc\" (UID: \"6c684495-6967-47c7-ba35-0b9b37110406\") " pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:02:41.624160 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:41.623880    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 21:02:41.624160 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.623895    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-etc-systemd\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.624160 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:41.623994    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs podName:3f6a3926-e94e-4981-9952-3ec7307a1e48 nodeName:}" failed. No retries permitted until 2026-07-20 21:02:42.123973895 +0000 UTC m=+3.027963464 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs") pod "network-metrics-daemon-khqfb" (UID: "3f6a3926-e94e-4981-9952-3ec7307a1e48") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 21:02:41.624160 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624108    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-run-systemd\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.624816 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624109    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-multus-conf-dir\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.624816 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624142    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/58fafef1-89fd-4094-9993-70ab07fecd85-host\") pod \"node-ca-8zfld\" (UID: \"58fafef1-89fd-4094-9993-70ab07fecd85\") " pod="openshift-image-registry/node-ca-8zfld"
Jul 20 21:02:41.624816 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624152    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-multus-conf-dir\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.624816 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624183    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/58fafef1-89fd-4094-9993-70ab07fecd85-host\") pod \"node-ca-8zfld\" (UID: \"58fafef1-89fd-4094-9993-70ab07fecd85\") " pod="openshift-image-registry/node-ca-8zfld"
Jul 20 21:02:41.624816 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624184    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c684495-6967-47c7-ba35-0b9b37110406-original-pull-secret\") pod \"global-pull-secret-syncer-g6ctc\" (UID: \"6c684495-6967-47c7-ba35-0b9b37110406\") " pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:02:41.624816 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624214    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-run\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.624816 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624220    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6c684495-6967-47c7-ba35-0b9b37110406-kubelet-config\") pod \"global-pull-secret-syncer-g6ctc\" (UID: \"6c684495-6967-47c7-ba35-0b9b37110406\") " pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:02:41.624816 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624237    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-var-lib-kubelet\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.624816 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624270    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1edc053e-71df-4f28-9552-0e7165b736d4-os-release\") pod \"multus-additional-cni-plugins-rzsh6\" (UID: \"1edc053e-71df-4f28-9552-0e7165b736d4\") " pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:41.624816 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:41.624280    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 21:02:41.624816 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624295    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-host-kubelet\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.624816 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:41.624335    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c684495-6967-47c7-ba35-0b9b37110406-original-pull-secret podName:6c684495-6967-47c7-ba35-0b9b37110406 nodeName:}" failed. No retries permitted until 2026-07-20 21:02:42.124315623 +0000 UTC m=+3.028305184 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c684495-6967-47c7-ba35-0b9b37110406-original-pull-secret") pod "global-pull-secret-syncer-g6ctc" (UID: "6c684495-6967-47c7-ba35-0b9b37110406") : object "kube-system"/"original-pull-secret" not registered
Jul 20 21:02:41.624816 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624354    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-host-kubelet\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.624816 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624355    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-etc-openvswitch\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.625451 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624388    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-etc-openvswitch\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.625451 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624389    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-system-cni-dir\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.625451 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624436    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-run-ovn\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.625451 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624454    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-system-cni-dir\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.625451 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624462    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-node-log\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.625451 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624479    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-host-run-ovn-kubernetes\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.625451 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624498    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-host\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.625451 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624501    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-run-ovn\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.625451 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624522    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/31d116a1-5993-427a-b6e2-d500f2a38c00-iptables-alerter-script\") pod \"iptables-alerter-wt9j5\" (UID: \"31d116a1-5993-427a-b6e2-d500f2a38c00\") " pod="openshift-network-operator/iptables-alerter-wt9j5"
Jul 20 21:02:41.625451 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624529    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-node-log\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.625451 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624549    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c11a809e-92e3-4887-9ca9-c1c3617c3c6c-socket-dir\") pod \"aws-ebs-csi-driver-node-8k9vf\" (UID: \"c11a809e-92e3-4887-9ca9-c1c3617c3c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf"
Jul 20 21:02:41.625451 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624584    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c11a809e-92e3-4887-9ca9-c1c3617c3c6c-registration-dir\") pod \"aws-ebs-csi-driver-node-8k9vf\" (UID: \"c11a809e-92e3-4887-9ca9-c1c3617c3c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf"
Jul 20 21:02:41.625451 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624610    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-host-run-k8s-cni-cncf-io\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.625451 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624636    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a1111bd9-7008-4387-8e06-83019291010b-multus-daemon-config\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.626124 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624669    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-log-socket\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.626124 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624699    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-etc-sysconfig\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.626124 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624718    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bf582cd6-b53c-45c1-b06b-8eea95f85a95-konnectivity-ca\") pod \"konnectivity-agent-bpvws\" (UID: \"bf582cd6-b53c-45c1-b06b-8eea95f85a95\") " pod="kube-system/konnectivity-agent-bpvws"
Jul 20 21:02:41.626124 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624740    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1edc053e-71df-4f28-9552-0e7165b736d4-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzsh6\" (UID: \"1edc053e-71df-4f28-9552-0e7165b736d4\") " pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:41.626124 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624760    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwh8h\" (UniqueName: \"kubernetes.io/projected/7f5aea7c-8d2c-4073-af50-ac30377c7e98-kube-api-access-rwh8h\") pod \"network-check-target-mr5nf\" (UID: \"7f5aea7c-8d2c-4073-af50-ac30377c7e98\") " pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:02:41.626124 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624788    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-host-run-netns\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.626124 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624814    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a1111bd9-7008-4387-8e06-83019291010b-cni-binary-copy\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.626124 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624840    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78npv\" (UniqueName: \"kubernetes.io/projected/0a812cf7-8e6f-48e2-a22a-29539fa392da-kube-api-access-78npv\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.626124 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624865    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06fb01b0-b8c5-45c2-b683-7aeec6af3796-tmp\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.626124 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624889    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-multus-socket-dir-parent\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.626124 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624914    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-host-run-netns\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.626124 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624939    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsv88\" (UniqueName: \"kubernetes.io/projected/a1111bd9-7008-4387-8e06-83019291010b-kube-api-access-hsv88\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.626124 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624961    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/06fb01b0-b8c5-45c2-b683-7aeec6af3796-etc-tuned\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.626124 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624985    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24q2q\" (UniqueName: \"kubernetes.io/projected/bcabbaa9-9e52-49f0-8786-ae54a37210e3-kube-api-access-24q2q\") pod \"node-resolver-7k6k7\" (UID: \"bcabbaa9-9e52-49f0-8786-ae54a37210e3\") " pod="openshift-dns/node-resolver-7k6k7"
Jul 20 21:02:41.626768 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625010    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0a812cf7-8e6f-48e2-a22a-29539fa392da-env-overrides\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.626768 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625035    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0a812cf7-8e6f-48e2-a22a-29539fa392da-ovn-node-metrics-cert\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.626768 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625058    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-etc-sysctl-d\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.626768 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625097    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-etc-sysctl-conf\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.626768 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625099    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-host-run-k8s-cni-cncf-io\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.626768 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625120    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bf582cd6-b53c-45c1-b06b-8eea95f85a95-agent-certs\") pod \"konnectivity-agent-bpvws\" (UID: \"bf582cd6-b53c-45c1-b06b-8eea95f85a95\") " pod="kube-system/konnectivity-agent-bpvws"
Jul 20 21:02:41.626768 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.624550    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-host-run-ovn-kubernetes\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.626768 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625143    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bcabbaa9-9e52-49f0-8786-ae54a37210e3-hosts-file\") pod \"node-resolver-7k6k7\" (UID: \"bcabbaa9-9e52-49f0-8786-ae54a37210e3\") " pod="openshift-dns/node-resolver-7k6k7"
Jul 20 21:02:41.626768 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625160    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0a812cf7-8e6f-48e2-a22a-29539fa392da-ovnkube-config\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.626768 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625172    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-log-socket\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.626768 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625184    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a1111bd9-7008-4387-8e06-83019291010b-multus-daemon-config\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.626768 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625207    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0a812cf7-8e6f-48e2-a22a-29539fa392da-ovnkube-script-lib\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.626768 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625232    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd9mx\" (UniqueName: \"kubernetes.io/projected/3f6a3926-e94e-4981-9952-3ec7307a1e48-kube-api-access-jd9mx\") pod \"network-metrics-daemon-khqfb\" (UID: \"3f6a3926-e94e-4981-9952-3ec7307a1e48\") " pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:02:41.626768 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625249    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-lib-modules\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.627423 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625265    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-host-var-lib-kubelet\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.627423 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625280    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-host-slash\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.627423 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625299    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-host-cni-netd\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.627423 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625324    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-sys\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.627423 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625339    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1edc053e-71df-4f28-9552-0e7165b736d4-system-cni-dir\") pod \"multus-additional-cni-plugins-rzsh6\" (UID: \"1edc053e-71df-4f28-9552-0e7165b736d4\") " pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:41.627423 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625356    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-host-cni-bin\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.627423 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625372    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6c684495-6967-47c7-ba35-0b9b37110406-dbus\") pod \"global-pull-secret-syncer-g6ctc\" (UID: \"6c684495-6967-47c7-ba35-0b9b37110406\") " pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:02:41.627423 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625393    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c11a809e-92e3-4887-9ca9-c1c3617c3c6c-sys-fs\") pod \"aws-ebs-csi-driver-node-8k9vf\" (UID: \"c11a809e-92e3-4887-9ca9-c1c3617c3c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf"
Jul 20 21:02:41.627423 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625419    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn9xx\" (UniqueName: \"kubernetes.io/projected/c11a809e-92e3-4887-9ca9-c1c3617c3c6c-kube-api-access-xn9xx\") pod \"aws-ebs-csi-driver-node-8k9vf\" (UID: \"c11a809e-92e3-4887-9ca9-c1c3617c3c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf"
Jul 20 21:02:41.627423 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625445    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-multus-cni-dir\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.627423 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625469    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-host-run-multus-certs\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.627423 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625492    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-etc-modprobe-d\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.627423 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625508    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7xdq\" (UniqueName: \"kubernetes.io/projected/06fb01b0-b8c5-45c2-b683-7aeec6af3796-kube-api-access-n7xdq\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.627423 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625525    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1edc053e-71df-4f28-9552-0e7165b736d4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzsh6\" (UID: \"1edc053e-71df-4f28-9552-0e7165b736d4\") " pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:41.627977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625541    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1edc053e-71df-4f28-9552-0e7165b736d4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzsh6\" (UID: \"1edc053e-71df-4f28-9552-0e7165b736d4\") " pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:41.627977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625566    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-cnibin\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.627977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625589    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0a812cf7-8e6f-48e2-a22a-29539fa392da-env-overrides\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.627977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625596    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-os-release\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.627977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625620    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-host-var-lib-cni-bin\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.627977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625644    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-host-var-lib-cni-multus\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.627977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625650    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a1111bd9-7008-4387-8e06-83019291010b-cni-binary-copy\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.627977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625668    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-hostroot\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.627977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625741    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-multus-socket-dir-parent\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.627977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625736    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-hostroot\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.627977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625798    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-host-run-netns\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.627977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.626110    2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 20 21:02:41.627977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.626228    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0a812cf7-8e6f-48e2-a22a-29539fa392da-ovnkube-config\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.627977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.625234    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-host-run-netns\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.627977 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.626300    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-host-var-lib-kubelet\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.628712 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.626317    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-cnibin\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.628712 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.626345    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-host-slash\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.628712 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.626365    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-os-release\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.628712 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.626372    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-host-cni-netd\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.628712 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.626400    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-host-var-lib-cni-bin\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.628712 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.626410    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0a812cf7-8e6f-48e2-a22a-29539fa392da-host-cni-bin\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.628712 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.626445    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-host-var-lib-cni-multus\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.628712 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.626477    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-host-run-multus-certs\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.628712 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.626502    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6c684495-6967-47c7-ba35-0b9b37110406-dbus\") pod \"global-pull-secret-syncer-g6ctc\" (UID: \"6c684495-6967-47c7-ba35-0b9b37110406\") " pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:02:41.628712 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.626532    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a1111bd9-7008-4387-8e06-83019291010b-multus-cni-dir\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.628712 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.626610    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0a812cf7-8e6f-48e2-a22a-29539fa392da-ovnkube-script-lib\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.630556 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.630537    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0a812cf7-8e6f-48e2-a22a-29539fa392da-ovn-node-metrics-cert\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.634490 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.634431    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n62fq\" (UniqueName: \"kubernetes.io/projected/58fafef1-89fd-4094-9993-70ab07fecd85-kube-api-access-n62fq\") pod \"node-ca-8zfld\" (UID: \"58fafef1-89fd-4094-9993-70ab07fecd85\") " pod="openshift-image-registry/node-ca-8zfld"
Jul 20 21:02:41.635605 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.635551    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78npv\" (UniqueName: \"kubernetes.io/projected/0a812cf7-8e6f-48e2-a22a-29539fa392da-kube-api-access-78npv\") pod \"ovnkube-node-w6bqv\" (UID: \"0a812cf7-8e6f-48e2-a22a-29539fa392da\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.636199 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.636140    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsv88\" (UniqueName: \"kubernetes.io/projected/a1111bd9-7008-4387-8e06-83019291010b-kube-api-access-hsv88\") pod \"multus-4hzwh\" (UID: \"a1111bd9-7008-4387-8e06-83019291010b\") " pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.637830 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.637811    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd9mx\" (UniqueName: \"kubernetes.io/projected/3f6a3926-e94e-4981-9952-3ec7307a1e48-kube-api-access-jd9mx\") pod \"network-metrics-daemon-khqfb\" (UID: \"3f6a3926-e94e-4981-9952-3ec7307a1e48\") " pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:02:41.672492 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.672463    2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 21:02:41.723918 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.723865    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-253.ec2.internal" event={"ID":"daa60d7c62b18ac55a74c9a2a524cd8f","Type":"ContainerStarted","Data":"df22ee010e0df6c85853f2e35e9d7d1911bec47e06c48c941754150ea768ae7b"}
Jul 20 21:02:41.724893 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.724874    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal" event={"ID":"4cd988cd2c2cd61004aa8c00d8ce4a16","Type":"ContainerStarted","Data":"2afc0bc1e2067a2d135f5c2ec43de60226c46c750e5684378bb9b9e630a87749"}
Jul 20 21:02:41.726391 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726361    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-host\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.726482 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726393    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/31d116a1-5993-427a-b6e2-d500f2a38c00-iptables-alerter-script\") pod \"iptables-alerter-wt9j5\" (UID: \"31d116a1-5993-427a-b6e2-d500f2a38c00\") " pod="openshift-network-operator/iptables-alerter-wt9j5"
Jul 20 21:02:41.726482 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726417    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c11a809e-92e3-4887-9ca9-c1c3617c3c6c-socket-dir\") pod \"aws-ebs-csi-driver-node-8k9vf\" (UID: \"c11a809e-92e3-4887-9ca9-c1c3617c3c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf"
Jul 20 21:02:41.726482 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726439    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c11a809e-92e3-4887-9ca9-c1c3617c3c6c-registration-dir\") pod \"aws-ebs-csi-driver-node-8k9vf\" (UID: \"c11a809e-92e3-4887-9ca9-c1c3617c3c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf"
Jul 20 21:02:41.726482 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726464    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-etc-sysconfig\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.726677 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726487    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bf582cd6-b53c-45c1-b06b-8eea95f85a95-konnectivity-ca\") pod \"konnectivity-agent-bpvws\" (UID: \"bf582cd6-b53c-45c1-b06b-8eea95f85a95\") " pod="kube-system/konnectivity-agent-bpvws"
Jul 20 21:02:41.726677 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726493    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-host\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.726677 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726511    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1edc053e-71df-4f28-9552-0e7165b736d4-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzsh6\" (UID: \"1edc053e-71df-4f28-9552-0e7165b736d4\") " pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:41.726677 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726561    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-etc-sysconfig\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.726677 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726559    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c11a809e-92e3-4887-9ca9-c1c3617c3c6c-registration-dir\") pod \"aws-ebs-csi-driver-node-8k9vf\" (UID: \"c11a809e-92e3-4887-9ca9-c1c3617c3c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf"
Jul 20 21:02:41.726677 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726589    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwh8h\" (UniqueName: \"kubernetes.io/projected/7f5aea7c-8d2c-4073-af50-ac30377c7e98-kube-api-access-rwh8h\") pod \"network-check-target-mr5nf\" (UID: \"7f5aea7c-8d2c-4073-af50-ac30377c7e98\") " pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:02:41.726677 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726622    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c11a809e-92e3-4887-9ca9-c1c3617c3c6c-socket-dir\") pod \"aws-ebs-csi-driver-node-8k9vf\" (UID: \"c11a809e-92e3-4887-9ca9-c1c3617c3c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf"
Jul 20 21:02:41.726677 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726626    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06fb01b0-b8c5-45c2-b683-7aeec6af3796-tmp\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.726677 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726673    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/06fb01b0-b8c5-45c2-b683-7aeec6af3796-etc-tuned\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.727162 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726700    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24q2q\" (UniqueName: \"kubernetes.io/projected/bcabbaa9-9e52-49f0-8786-ae54a37210e3-kube-api-access-24q2q\") pod \"node-resolver-7k6k7\" (UID: \"bcabbaa9-9e52-49f0-8786-ae54a37210e3\") " pod="openshift-dns/node-resolver-7k6k7"
Jul 20 21:02:41.727162 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726726    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-etc-sysctl-d\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.727162 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726751    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-etc-sysctl-conf\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.727162 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726772    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bf582cd6-b53c-45c1-b06b-8eea95f85a95-agent-certs\") pod \"konnectivity-agent-bpvws\" (UID: \"bf582cd6-b53c-45c1-b06b-8eea95f85a95\") " pod="kube-system/konnectivity-agent-bpvws"
Jul 20 21:02:41.727162 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726795    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bcabbaa9-9e52-49f0-8786-ae54a37210e3-hosts-file\") pod \"node-resolver-7k6k7\" (UID: \"bcabbaa9-9e52-49f0-8786-ae54a37210e3\") " pod="openshift-dns/node-resolver-7k6k7"
Jul 20 21:02:41.727162 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726820    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-lib-modules\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.727162 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726846    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-sys\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.727162 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726870    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1edc053e-71df-4f28-9552-0e7165b736d4-system-cni-dir\") pod \"multus-additional-cni-plugins-rzsh6\" (UID: \"1edc053e-71df-4f28-9552-0e7165b736d4\") " pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:41.727162 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726895    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c11a809e-92e3-4887-9ca9-c1c3617c3c6c-sys-fs\") pod \"aws-ebs-csi-driver-node-8k9vf\" (UID: \"c11a809e-92e3-4887-9ca9-c1c3617c3c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf"
Jul 20 21:02:41.727162 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726920    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn9xx\" (UniqueName: \"kubernetes.io/projected/c11a809e-92e3-4887-9ca9-c1c3617c3c6c-kube-api-access-xn9xx\") pod \"aws-ebs-csi-driver-node-8k9vf\" (UID: \"c11a809e-92e3-4887-9ca9-c1c3617c3c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf"
Jul 20 21:02:41.727162 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726944    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-etc-modprobe-d\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.727162 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726970    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7xdq\" (UniqueName: \"kubernetes.io/projected/06fb01b0-b8c5-45c2-b683-7aeec6af3796-kube-api-access-n7xdq\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.727162 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.726993    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1edc053e-71df-4f28-9552-0e7165b736d4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzsh6\" (UID: \"1edc053e-71df-4f28-9552-0e7165b736d4\") " pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:41.727162 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727015    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-etc-sysctl-conf\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.727878 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727021    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1edc053e-71df-4f28-9552-0e7165b736d4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzsh6\" (UID: \"1edc053e-71df-4f28-9552-0e7165b736d4\") " pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:41.727878 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727099    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-etc-kubernetes\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.727878 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727125    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcabbaa9-9e52-49f0-8786-ae54a37210e3-tmp-dir\") pod \"node-resolver-7k6k7\" (UID: \"bcabbaa9-9e52-49f0-8786-ae54a37210e3\") " pod="openshift-dns/node-resolver-7k6k7"
Jul 20 21:02:41.727878 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727134    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/31d116a1-5993-427a-b6e2-d500f2a38c00-iptables-alerter-script\") pod \"iptables-alerter-wt9j5\" (UID: \"31d116a1-5993-427a-b6e2-d500f2a38c00\") " pod="openshift-network-operator/iptables-alerter-wt9j5"
Jul 20 21:02:41.727878 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727150    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whxxn\" (UniqueName: \"kubernetes.io/projected/31d116a1-5993-427a-b6e2-d500f2a38c00-kube-api-access-whxxn\") pod \"iptables-alerter-wt9j5\" (UID: \"31d116a1-5993-427a-b6e2-d500f2a38c00\") " pod="openshift-network-operator/iptables-alerter-wt9j5"
Jul 20 21:02:41.727878 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727173    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1edc053e-71df-4f28-9552-0e7165b736d4-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzsh6\" (UID: \"1edc053e-71df-4f28-9552-0e7165b736d4\") " pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:41.727878 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727181    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bf582cd6-b53c-45c1-b06b-8eea95f85a95-konnectivity-ca\") pod \"konnectivity-agent-bpvws\" (UID: \"bf582cd6-b53c-45c1-b06b-8eea95f85a95\") " pod="kube-system/konnectivity-agent-bpvws"
Jul 20 21:02:41.727878 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727178    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c11a809e-92e3-4887-9ca9-c1c3617c3c6c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8k9vf\" (UID: \"c11a809e-92e3-4887-9ca9-c1c3617c3c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf"
Jul 20 21:02:41.727878 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727221    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c11a809e-92e3-4887-9ca9-c1c3617c3c6c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8k9vf\" (UID: \"c11a809e-92e3-4887-9ca9-c1c3617c3c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf"
Jul 20 21:02:41.727878 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727228    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c11a809e-92e3-4887-9ca9-c1c3617c3c6c-device-dir\") pod \"aws-ebs-csi-driver-node-8k9vf\" (UID: \"c11a809e-92e3-4887-9ca9-c1c3617c3c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf"
Jul 20 21:02:41.727878 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727240    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c11a809e-92e3-4887-9ca9-c1c3617c3c6c-sys-fs\") pod \"aws-ebs-csi-driver-node-8k9vf\" (UID: \"c11a809e-92e3-4887-9ca9-c1c3617c3c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf"
Jul 20 21:02:41.727878 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727253    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bcabbaa9-9e52-49f0-8786-ae54a37210e3-hosts-file\") pod \"node-resolver-7k6k7\" (UID: \"bcabbaa9-9e52-49f0-8786-ae54a37210e3\") " pod="openshift-dns/node-resolver-7k6k7"
Jul 20 21:02:41.727878 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727259    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1edc053e-71df-4f28-9552-0e7165b736d4-cnibin\") pod \"multus-additional-cni-plugins-rzsh6\" (UID: \"1edc053e-71df-4f28-9552-0e7165b736d4\") " pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:41.727878 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727286    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1edc053e-71df-4f28-9552-0e7165b736d4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzsh6\" (UID: \"1edc053e-71df-4f28-9552-0e7165b736d4\") " pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:41.728422 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727312    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9h54\" (UniqueName: \"kubernetes.io/projected/1edc053e-71df-4f28-9552-0e7165b736d4-kube-api-access-m9h54\") pod \"multus-additional-cni-plugins-rzsh6\" (UID: \"1edc053e-71df-4f28-9552-0e7165b736d4\") " pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:41.728422 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727355    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/31d116a1-5993-427a-b6e2-d500f2a38c00-host-slash\") pod \"iptables-alerter-wt9j5\" (UID: \"31d116a1-5993-427a-b6e2-d500f2a38c00\") " pod="openshift-network-operator/iptables-alerter-wt9j5"
Jul 20 21:02:41.728422 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727370    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-etc-modprobe-d\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.728422 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727379    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c11a809e-92e3-4887-9ca9-c1c3617c3c6c-etc-selinux\") pod \"aws-ebs-csi-driver-node-8k9vf\" (UID: \"c11a809e-92e3-4887-9ca9-c1c3617c3c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf"
Jul 20 21:02:41.728422 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727382    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-etc-sysctl-d\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.728422 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727469    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1edc053e-71df-4f28-9552-0e7165b736d4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzsh6\" (UID: \"1edc053e-71df-4f28-9552-0e7165b736d4\") " pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:41.728422 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727532    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-sys\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.728422 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727555    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/31d116a1-5993-427a-b6e2-d500f2a38c00-host-slash\") pod \"iptables-alerter-wt9j5\" (UID: \"31d116a1-5993-427a-b6e2-d500f2a38c00\") " pod="openshift-network-operator/iptables-alerter-wt9j5"
Jul 20 21:02:41.728422 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727604    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c11a809e-92e3-4887-9ca9-c1c3617c3c6c-etc-selinux\") pod \"aws-ebs-csi-driver-node-8k9vf\" (UID: \"c11a809e-92e3-4887-9ca9-c1c3617c3c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf"
Jul 20 21:02:41.728422 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727649    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-etc-kubernetes\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.728422 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727677    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcabbaa9-9e52-49f0-8786-ae54a37210e3-tmp-dir\") pod \"node-resolver-7k6k7\" (UID: \"bcabbaa9-9e52-49f0-8786-ae54a37210e3\") " pod="openshift-dns/node-resolver-7k6k7"
Jul 20 21:02:41.728422 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727685    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-etc-systemd\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.728422 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727742    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-etc-systemd\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.728422 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727787    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1edc053e-71df-4f28-9552-0e7165b736d4-cnibin\") pod \"multus-additional-cni-plugins-rzsh6\" (UID: \"1edc053e-71df-4f28-9552-0e7165b736d4\") " pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:41.728900 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727806    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c11a809e-92e3-4887-9ca9-c1c3617c3c6c-device-dir\") pod \"aws-ebs-csi-driver-node-8k9vf\" (UID: \"c11a809e-92e3-4887-9ca9-c1c3617c3c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf"
Jul 20 21:02:41.728900 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727822    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-lib-modules\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.728900 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727828    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1edc053e-71df-4f28-9552-0e7165b736d4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzsh6\" (UID: \"1edc053e-71df-4f28-9552-0e7165b736d4\") " pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:41.728900 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727199    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1edc053e-71df-4f28-9552-0e7165b736d4-system-cni-dir\") pod \"multus-additional-cni-plugins-rzsh6\" (UID: \"1edc053e-71df-4f28-9552-0e7165b736d4\") " pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:41.728900 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727874    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-run\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.728900 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727899    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-var-lib-kubelet\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.728900 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727923    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1edc053e-71df-4f28-9552-0e7165b736d4-os-release\") pod \"multus-additional-cni-plugins-rzsh6\" (UID: \"1edc053e-71df-4f28-9552-0e7165b736d4\") " pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:41.728900 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727930    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-run\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.728900 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727986    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06fb01b0-b8c5-45c2-b683-7aeec6af3796-var-lib-kubelet\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.728900 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.727993    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1edc053e-71df-4f28-9552-0e7165b736d4-os-release\") pod \"multus-additional-cni-plugins-rzsh6\" (UID: \"1edc053e-71df-4f28-9552-0e7165b736d4\") " pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:41.728900 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.728308    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1edc053e-71df-4f28-9552-0e7165b736d4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzsh6\" (UID: \"1edc053e-71df-4f28-9552-0e7165b736d4\") " pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:41.729457 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.729247    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06fb01b0-b8c5-45c2-b683-7aeec6af3796-tmp\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.729878 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.729827    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/06fb01b0-b8c5-45c2-b683-7aeec6af3796-etc-tuned\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.730120 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.730101    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bf582cd6-b53c-45c1-b06b-8eea95f85a95-agent-certs\") pod \"konnectivity-agent-bpvws\" (UID: \"bf582cd6-b53c-45c1-b06b-8eea95f85a95\") " pod="kube-system/konnectivity-agent-bpvws"
Jul 20 21:02:41.736577 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.736556    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn9xx\" (UniqueName: \"kubernetes.io/projected/c11a809e-92e3-4887-9ca9-c1c3617c3c6c-kube-api-access-xn9xx\") pod \"aws-ebs-csi-driver-node-8k9vf\" (UID: \"c11a809e-92e3-4887-9ca9-c1c3617c3c6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf"
Jul 20 21:02:41.737441 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:41.737325    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 21:02:41.737441 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:41.737369    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 21:02:41.737441 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:41.737381    2561 projected.go:196] Error preparing data for projected volume kube-api-access-rwh8h for pod openshift-network-diagnostics/network-check-target-mr5nf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 21:02:41.737636 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:41.737456    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f5aea7c-8d2c-4073-af50-ac30377c7e98-kube-api-access-rwh8h podName:7f5aea7c-8d2c-4073-af50-ac30377c7e98 nodeName:}" failed. No retries permitted until 2026-07-20 21:02:42.237435756 +0000 UTC m=+3.141425326 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rwh8h" (UniqueName: "kubernetes.io/projected/7f5aea7c-8d2c-4073-af50-ac30377c7e98-kube-api-access-rwh8h") pod "network-check-target-mr5nf" (UID: "7f5aea7c-8d2c-4073-af50-ac30377c7e98") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 21:02:41.739556 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.739509    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whxxn\" (UniqueName: \"kubernetes.io/projected/31d116a1-5993-427a-b6e2-d500f2a38c00-kube-api-access-whxxn\") pod \"iptables-alerter-wt9j5\" (UID: \"31d116a1-5993-427a-b6e2-d500f2a38c00\") " pod="openshift-network-operator/iptables-alerter-wt9j5"
Jul 20 21:02:41.739768 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.739742    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7xdq\" (UniqueName: \"kubernetes.io/projected/06fb01b0-b8c5-45c2-b683-7aeec6af3796-kube-api-access-n7xdq\") pod \"tuned-fhwv7\" (UID: \"06fb01b0-b8c5-45c2-b683-7aeec6af3796\") " pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.740126 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.740108    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24q2q\" (UniqueName: \"kubernetes.io/projected/bcabbaa9-9e52-49f0-8786-ae54a37210e3-kube-api-access-24q2q\") pod \"node-resolver-7k6k7\" (UID: \"bcabbaa9-9e52-49f0-8786-ae54a37210e3\") " pod="openshift-dns/node-resolver-7k6k7"
Jul 20 21:02:41.740184 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.740128    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9h54\" (UniqueName: \"kubernetes.io/projected/1edc053e-71df-4f28-9552-0e7165b736d4-kube-api-access-m9h54\") pod \"multus-additional-cni-plugins-rzsh6\" (UID: \"1edc053e-71df-4f28-9552-0e7165b736d4\") " pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:41.810217 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.810186    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4hzwh"
Jul 20 21:02:41.822369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.822338    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:02:41.841171 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.841145    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8zfld"
Jul 20 21:02:41.848190 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.848165    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wt9j5"
Jul 20 21:02:41.855173 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.855156    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fhwv7"
Jul 20 21:02:41.863099 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.863064    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bpvws"
Jul 20 21:02:41.869031 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.869010    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7k6k7"
Jul 20 21:02:41.876182 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.876159    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf"
Jul 20 21:02:41.882907 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:41.882887    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rzsh6"
Jul 20 21:02:42.131270 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:42.131185    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs\") pod \"network-metrics-daemon-khqfb\" (UID: \"3f6a3926-e94e-4981-9952-3ec7307a1e48\") " pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:02:42.131270 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:42.131224    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c684495-6967-47c7-ba35-0b9b37110406-original-pull-secret\") pod \"global-pull-secret-syncer-g6ctc\" (UID: \"6c684495-6967-47c7-ba35-0b9b37110406\") " pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:02:42.131437 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:42.131348    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 21:02:42.131437 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:42.131405    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c684495-6967-47c7-ba35-0b9b37110406-original-pull-secret podName:6c684495-6967-47c7-ba35-0b9b37110406 nodeName:}" failed. No retries permitted until 2026-07-20 21:02:43.131391869 +0000 UTC m=+4.035381426 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c684495-6967-47c7-ba35-0b9b37110406-original-pull-secret") pod "global-pull-secret-syncer-g6ctc" (UID: "6c684495-6967-47c7-ba35-0b9b37110406") : object "kube-system"/"original-pull-secret" not registered
Jul 20 21:02:42.131521 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:42.131349    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 21:02:42.131521 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:42.131488    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs podName:3f6a3926-e94e-4981-9952-3ec7307a1e48 nodeName:}" failed. No retries permitted until 2026-07-20 21:02:43.131474806 +0000 UTC m=+4.035464376 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs") pod "network-metrics-daemon-khqfb" (UID: "3f6a3926-e94e-4981-9952-3ec7307a1e48") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 21:02:42.333534 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:42.333498    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwh8h\" (UniqueName: \"kubernetes.io/projected/7f5aea7c-8d2c-4073-af50-ac30377c7e98-kube-api-access-rwh8h\") pod \"network-check-target-mr5nf\" (UID: \"7f5aea7c-8d2c-4073-af50-ac30377c7e98\") " pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:02:42.333690 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:42.333620    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 21:02:42.333690 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:42.333635    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 21:02:42.333690 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:42.333646    2561 projected.go:196] Error preparing data for projected volume kube-api-access-rwh8h for pod openshift-network-diagnostics/network-check-target-mr5nf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 21:02:42.333806 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:42.333693    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f5aea7c-8d2c-4073-af50-ac30377c7e98-kube-api-access-rwh8h podName:7f5aea7c-8d2c-4073-af50-ac30377c7e98 nodeName:}" failed. No retries permitted until 2026-07-20 21:02:43.333680873 +0000 UTC m=+4.237670429 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rwh8h" (UniqueName: "kubernetes.io/projected/7f5aea7c-8d2c-4073-af50-ac30377c7e98-kube-api-access-rwh8h") pod "network-check-target-mr5nf" (UID: "7f5aea7c-8d2c-4073-af50-ac30377c7e98") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 21:02:42.550114 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:42.550003    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 20:57:41 +0000 UTC" deadline="2028-01-04 15:13:09.676606777 +0000 UTC"
Jul 20 21:02:42.550114 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:42.550043    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12786h10m27.126568356s"
Jul 20 21:02:42.720862 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:42.720822    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:02:42.721030 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:42.720967    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr5nf" podUID="7f5aea7c-8d2c-4073-af50-ac30377c7e98"
Jul 20 21:02:42.772773 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:42.772679    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a812cf7_8e6f_48e2_a22a_29539fa392da.slice/crio-a7dee6b1e90a8429f3a0e15ecd8bda4a82e08375b37718b5ce0feb63ad83b505 WatchSource:0}: Error finding container a7dee6b1e90a8429f3a0e15ecd8bda4a82e08375b37718b5ce0feb63ad83b505: Status 404 returned error can't find the container with id a7dee6b1e90a8429f3a0e15ecd8bda4a82e08375b37718b5ce0feb63ad83b505
Jul 20 21:02:42.774128 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:42.774101    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc11a809e_92e3_4887_9ca9_c1c3617c3c6c.slice/crio-51169fddcf2867b5dbd38acd8838e4ca03db25ef9af910f1da644bf9f2c52420 WatchSource:0}: Error finding container 51169fddcf2867b5dbd38acd8838e4ca03db25ef9af910f1da644bf9f2c52420: Status 404 returned error can't find the container with id 51169fddcf2867b5dbd38acd8838e4ca03db25ef9af910f1da644bf9f2c52420
Jul 20 21:02:42.774941 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:42.774852    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06fb01b0_b8c5_45c2_b683_7aeec6af3796.slice/crio-78d5cda7e98826cf0711a8d4cd84ae5f2755637fd42b3a97cf3d67efdce90f64 WatchSource:0}: Error finding container 78d5cda7e98826cf0711a8d4cd84ae5f2755637fd42b3a97cf3d67efdce90f64: Status 404 returned error can't find the container with id 78d5cda7e98826cf0711a8d4cd84ae5f2755637fd42b3a97cf3d67efdce90f64
Jul 20 21:02:42.778410 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:42.778390    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1111bd9_7008_4387_8e06_83019291010b.slice/crio-086cb07b42b3c1a474d7b7d867d328fa522544cde53f0a2a5b1ef259de1ef271 WatchSource:0}: Error finding container 086cb07b42b3c1a474d7b7d867d328fa522544cde53f0a2a5b1ef259de1ef271: Status 404 returned error can't find the container with id 086cb07b42b3c1a474d7b7d867d328fa522544cde53f0a2a5b1ef259de1ef271
Jul 20 21:02:42.779203 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:42.779180    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1edc053e_71df_4f28_9552_0e7165b736d4.slice/crio-349baf0908bff9aef4ef8231bf08340640d172ca146eff1a7d2b4153c6cf5a91 WatchSource:0}: Error finding container 349baf0908bff9aef4ef8231bf08340640d172ca146eff1a7d2b4153c6cf5a91: Status 404 returned error can't find the container with id 349baf0908bff9aef4ef8231bf08340640d172ca146eff1a7d2b4153c6cf5a91
Jul 20 21:02:42.780066 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:42.780025    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58fafef1_89fd_4094_9993_70ab07fecd85.slice/crio-eeb5d4e0ac940ced9d4e36caadcb3ce6aca7e500324256996d42ded19178e931 WatchSource:0}: Error finding container eeb5d4e0ac940ced9d4e36caadcb3ce6aca7e500324256996d42ded19178e931: Status 404 returned error can't find the container with id eeb5d4e0ac940ced9d4e36caadcb3ce6aca7e500324256996d42ded19178e931
Jul 20 21:02:42.783865 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:42.783088    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31d116a1_5993_427a_b6e2_d500f2a38c00.slice/crio-df38fd03d0582b8b96402b2e1c80cba93c3d1d84d6959312b3979a19d06f6ff4 WatchSource:0}: Error finding container df38fd03d0582b8b96402b2e1c80cba93c3d1d84d6959312b3979a19d06f6ff4: Status 404 returned error can't find the container with id df38fd03d0582b8b96402b2e1c80cba93c3d1d84d6959312b3979a19d06f6ff4
Jul 20 21:02:42.786422 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:02:42.786203    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcabbaa9_9e52_49f0_8786_ae54a37210e3.slice/crio-0aa5c4c80ea28f41f9ed5148c11e1f5c29d6b7983dae9ca8cf58cc9429090488 WatchSource:0}: Error finding container 0aa5c4c80ea28f41f9ed5148c11e1f5c29d6b7983dae9ca8cf58cc9429090488: Status 404 returned error can't find the container with id 0aa5c4c80ea28f41f9ed5148c11e1f5c29d6b7983dae9ca8cf58cc9429090488
Jul 20 21:02:43.139504 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:43.139266    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs\") pod \"network-metrics-daemon-khqfb\" (UID: \"3f6a3926-e94e-4981-9952-3ec7307a1e48\") " pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:02:43.139504 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:43.139473    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c684495-6967-47c7-ba35-0b9b37110406-original-pull-secret\") pod \"global-pull-secret-syncer-g6ctc\" (UID: \"6c684495-6967-47c7-ba35-0b9b37110406\") " pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:02:43.139671 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:43.139415    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 21:02:43.139671 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:43.139568    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 21:02:43.139671 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:43.139602    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs podName:3f6a3926-e94e-4981-9952-3ec7307a1e48 nodeName:}" failed. No retries permitted until 2026-07-20 21:02:45.139579199 +0000 UTC m=+6.043568772 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs") pod "network-metrics-daemon-khqfb" (UID: "3f6a3926-e94e-4981-9952-3ec7307a1e48") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 21:02:43.139671 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:43.139622    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c684495-6967-47c7-ba35-0b9b37110406-original-pull-secret podName:6c684495-6967-47c7-ba35-0b9b37110406 nodeName:}" failed. No retries permitted until 2026-07-20 21:02:45.139615014 +0000 UTC m=+6.043604571 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c684495-6967-47c7-ba35-0b9b37110406-original-pull-secret") pod "global-pull-secret-syncer-g6ctc" (UID: "6c684495-6967-47c7-ba35-0b9b37110406") : object "kube-system"/"original-pull-secret" not registered
Jul 20 21:02:43.341216 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:43.341172    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwh8h\" (UniqueName: \"kubernetes.io/projected/7f5aea7c-8d2c-4073-af50-ac30377c7e98-kube-api-access-rwh8h\") pod \"network-check-target-mr5nf\" (UID: \"7f5aea7c-8d2c-4073-af50-ac30377c7e98\") " pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:02:43.341449 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:43.341414    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 21:02:43.341449 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:43.341443    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 21:02:43.341567 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:43.341458    2561 projected.go:196] Error preparing data for projected volume kube-api-access-rwh8h for pod openshift-network-diagnostics/network-check-target-mr5nf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 21:02:43.341567 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:43.341525    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f5aea7c-8d2c-4073-af50-ac30377c7e98-kube-api-access-rwh8h podName:7f5aea7c-8d2c-4073-af50-ac30377c7e98 nodeName:}" failed. No retries permitted until 2026-07-20 21:02:45.341503952 +0000 UTC m=+6.245493523 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rwh8h" (UniqueName: "kubernetes.io/projected/7f5aea7c-8d2c-4073-af50-ac30377c7e98-kube-api-access-rwh8h") pod "network-check-target-mr5nf" (UID: "7f5aea7c-8d2c-4073-af50-ac30377c7e98") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 21:02:43.551245 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:43.551134    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 20:57:41 +0000 UTC" deadline="2027-12-25 07:14:46.020474125 +0000 UTC"
Jul 20 21:02:43.551245 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:43.551198    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12538h12m2.469280078s"
Jul 20 21:02:43.722401 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:43.722371    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:02:43.722562 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:43.722502    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6ctc" podUID="6c684495-6967-47c7-ba35-0b9b37110406"
Jul 20 21:02:43.723441 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:43.722902    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:02:43.723441 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:43.723019    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khqfb" podUID="3f6a3926-e94e-4981-9952-3ec7307a1e48"
Jul 20 21:02:43.747177 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:43.747140    2561 generic.go:356] "Generic (PLEG): container finished" podID="4cd988cd2c2cd61004aa8c00d8ce4a16" containerID="968649fc47ba542f376dc672b61e6d87429f8cb01309424a44042a394bc44dfc" exitCode=0
Jul 20 21:02:43.747325 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:43.747237    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal" event={"ID":"4cd988cd2c2cd61004aa8c00d8ce4a16","Type":"ContainerDied","Data":"968649fc47ba542f376dc672b61e6d87429f8cb01309424a44042a394bc44dfc"}
Jul 20 21:02:43.765153 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:43.765087    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7k6k7" event={"ID":"bcabbaa9-9e52-49f0-8786-ae54a37210e3","Type":"ContainerStarted","Data":"0aa5c4c80ea28f41f9ed5148c11e1f5c29d6b7983dae9ca8cf58cc9429090488"}
Jul 20 21:02:43.779888 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:43.779431    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8zfld" event={"ID":"58fafef1-89fd-4094-9993-70ab07fecd85","Type":"ContainerStarted","Data":"eeb5d4e0ac940ced9d4e36caadcb3ce6aca7e500324256996d42ded19178e931"}
Jul 20 21:02:43.788745 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:43.788715    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bpvws" event={"ID":"bf582cd6-b53c-45c1-b06b-8eea95f85a95","Type":"ContainerStarted","Data":"85e15c6e7688ed6bbfeb27b57f2bd9b2e569e3d3b18f010c2a9eab8a97eee7ab"}
Jul 20 21:02:43.811730 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:43.811692    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fhwv7" event={"ID":"06fb01b0-b8c5-45c2-b683-7aeec6af3796","Type":"ContainerStarted","Data":"78d5cda7e98826cf0711a8d4cd84ae5f2755637fd42b3a97cf3d67efdce90f64"}
Jul 20 21:02:43.816405 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:43.816378    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf" event={"ID":"c11a809e-92e3-4887-9ca9-c1c3617c3c6c","Type":"ContainerStarted","Data":"51169fddcf2867b5dbd38acd8838e4ca03db25ef9af910f1da644bf9f2c52420"}
Jul 20 21:02:43.820590 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:43.820563    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv" event={"ID":"0a812cf7-8e6f-48e2-a22a-29539fa392da","Type":"ContainerStarted","Data":"a7dee6b1e90a8429f3a0e15ecd8bda4a82e08375b37718b5ce0feb63ad83b505"}
Jul 20 21:02:43.836681 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:43.836646    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-253.ec2.internal" event={"ID":"daa60d7c62b18ac55a74c9a2a524cd8f","Type":"ContainerStarted","Data":"06f9c9f0203d80a142affac6374bf39e8fcbe09f14dee1cafdd0e52ad1a9161c"}
Jul 20 21:02:43.843008 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:43.842850    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wt9j5" event={"ID":"31d116a1-5993-427a-b6e2-d500f2a38c00","Type":"ContainerStarted","Data":"df38fd03d0582b8b96402b2e1c80cba93c3d1d84d6959312b3979a19d06f6ff4"}
Jul 20 21:02:43.850048 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:43.850018    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzsh6" event={"ID":"1edc053e-71df-4f28-9552-0e7165b736d4","Type":"ContainerStarted","Data":"349baf0908bff9aef4ef8231bf08340640d172ca146eff1a7d2b4153c6cf5a91"}
Jul 20 21:02:43.862600 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:43.862544    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4hzwh" event={"ID":"a1111bd9-7008-4387-8e06-83019291010b","Type":"ContainerStarted","Data":"086cb07b42b3c1a474d7b7d867d328fa522544cde53f0a2a5b1ef259de1ef271"}
Jul 20 21:02:44.720768 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:44.720735    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:02:44.721258 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:44.720869    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr5nf" podUID="7f5aea7c-8d2c-4073-af50-ac30377c7e98"
Jul 20 21:02:44.886189 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:44.886105    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal" event={"ID":"4cd988cd2c2cd61004aa8c00d8ce4a16","Type":"ContainerStarted","Data":"17edb461dda09cf42db6e62836447d36361f7fe0dbaec8775fb705f1023192a7"}
Jul 20 21:02:44.901791 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:44.901732    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-253.ec2.internal" podStartSLOduration=4.901708641 podStartE2EDuration="4.901708641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 21:02:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:02:43.855829723 +0000 UTC m=+4.759819302" watchObservedRunningTime="2026-07-20 21:02:44.901708641 +0000 UTC m=+5.805698219"
Jul 20 21:02:45.157410 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:45.157245    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs\") pod \"network-metrics-daemon-khqfb\" (UID: \"3f6a3926-e94e-4981-9952-3ec7307a1e48\") " pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:02:45.157410 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:45.157302    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c684495-6967-47c7-ba35-0b9b37110406-original-pull-secret\") pod \"global-pull-secret-syncer-g6ctc\" (UID: \"6c684495-6967-47c7-ba35-0b9b37110406\") " pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:02:45.157613 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:45.157426    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 21:02:45.157613 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:45.157489    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c684495-6967-47c7-ba35-0b9b37110406-original-pull-secret podName:6c684495-6967-47c7-ba35-0b9b37110406 nodeName:}" failed. No retries permitted until 2026-07-20 21:02:49.157471525 +0000 UTC m=+10.061461086 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c684495-6967-47c7-ba35-0b9b37110406-original-pull-secret") pod "global-pull-secret-syncer-g6ctc" (UID: "6c684495-6967-47c7-ba35-0b9b37110406") : object "kube-system"/"original-pull-secret" not registered
Jul 20 21:02:45.157875 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:45.157855    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 21:02:45.157949 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:45.157910    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs podName:3f6a3926-e94e-4981-9952-3ec7307a1e48 nodeName:}" failed. No retries permitted until 2026-07-20 21:02:49.157895137 +0000 UTC m=+10.061884698 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs") pod "network-metrics-daemon-khqfb" (UID: "3f6a3926-e94e-4981-9952-3ec7307a1e48") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 21:02:45.360277 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:45.359626    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwh8h\" (UniqueName: \"kubernetes.io/projected/7f5aea7c-8d2c-4073-af50-ac30377c7e98-kube-api-access-rwh8h\") pod \"network-check-target-mr5nf\" (UID: \"7f5aea7c-8d2c-4073-af50-ac30377c7e98\") " pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:02:45.360277 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:45.359816    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 21:02:45.360277 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:45.359842    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 21:02:45.360277 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:45.359855    2561 projected.go:196] Error preparing data for projected volume kube-api-access-rwh8h for pod openshift-network-diagnostics/network-check-target-mr5nf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 21:02:45.360277 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:45.359915    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f5aea7c-8d2c-4073-af50-ac30377c7e98-kube-api-access-rwh8h podName:7f5aea7c-8d2c-4073-af50-ac30377c7e98 nodeName:}" failed. No retries permitted until 2026-07-20 21:02:49.359897412 +0000 UTC m=+10.263886993 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rwh8h" (UniqueName: "kubernetes.io/projected/7f5aea7c-8d2c-4073-af50-ac30377c7e98-kube-api-access-rwh8h") pod "network-check-target-mr5nf" (UID: "7f5aea7c-8d2c-4073-af50-ac30377c7e98") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 21:02:45.721673 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:45.721180    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:02:45.721673 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:45.721317    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6ctc" podUID="6c684495-6967-47c7-ba35-0b9b37110406"
Jul 20 21:02:45.724130 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:45.722338    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:02:45.724130 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:45.722474    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khqfb" podUID="3f6a3926-e94e-4981-9952-3ec7307a1e48"
Jul 20 21:02:46.720254 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:46.720180    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:02:46.720431 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:46.720381    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr5nf" podUID="7f5aea7c-8d2c-4073-af50-ac30377c7e98"
Jul 20 21:02:47.721027 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:47.720809    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:02:47.721027 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:47.720808    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:02:47.721027 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:47.720965    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khqfb" podUID="3f6a3926-e94e-4981-9952-3ec7307a1e48"
Jul 20 21:02:47.721027 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:47.720987    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6ctc" podUID="6c684495-6967-47c7-ba35-0b9b37110406"
Jul 20 21:02:48.720603 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:48.720572    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:02:48.720807 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:48.720672    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr5nf" podUID="7f5aea7c-8d2c-4073-af50-ac30377c7e98"
Jul 20 21:02:49.189525 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:49.189364    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs\") pod \"network-metrics-daemon-khqfb\" (UID: \"3f6a3926-e94e-4981-9952-3ec7307a1e48\") " pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:02:49.189525 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:49.189428    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c684495-6967-47c7-ba35-0b9b37110406-original-pull-secret\") pod \"global-pull-secret-syncer-g6ctc\" (UID: \"6c684495-6967-47c7-ba35-0b9b37110406\") " pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:02:49.189525 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:49.189491    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 21:02:49.190137 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:49.189560    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 21:02:49.190137 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:49.189571    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs podName:3f6a3926-e94e-4981-9952-3ec7307a1e48 nodeName:}" failed. No retries permitted until 2026-07-20 21:02:57.189548323 +0000 UTC m=+18.093537974 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs") pod "network-metrics-daemon-khqfb" (UID: "3f6a3926-e94e-4981-9952-3ec7307a1e48") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 21:02:49.190137 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:49.189619    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c684495-6967-47c7-ba35-0b9b37110406-original-pull-secret podName:6c684495-6967-47c7-ba35-0b9b37110406 nodeName:}" failed. No retries permitted until 2026-07-20 21:02:57.189601765 +0000 UTC m=+18.093591334 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c684495-6967-47c7-ba35-0b9b37110406-original-pull-secret") pod "global-pull-secret-syncer-g6ctc" (UID: "6c684495-6967-47c7-ba35-0b9b37110406") : object "kube-system"/"original-pull-secret" not registered
Jul 20 21:02:49.391394 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:49.391355    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwh8h\" (UniqueName: \"kubernetes.io/projected/7f5aea7c-8d2c-4073-af50-ac30377c7e98-kube-api-access-rwh8h\") pod \"network-check-target-mr5nf\" (UID: \"7f5aea7c-8d2c-4073-af50-ac30377c7e98\") " pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:02:49.391578 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:49.391553    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 21:02:49.391578 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:49.391577    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 21:02:49.391688 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:49.391589    2561 projected.go:196] Error preparing data for projected volume kube-api-access-rwh8h for pod openshift-network-diagnostics/network-check-target-mr5nf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 21:02:49.391688 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:49.391657    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f5aea7c-8d2c-4073-af50-ac30377c7e98-kube-api-access-rwh8h podName:7f5aea7c-8d2c-4073-af50-ac30377c7e98 nodeName:}" failed. No retries permitted until 2026-07-20 21:02:57.391637496 +0000 UTC m=+18.295627067 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rwh8h" (UniqueName: "kubernetes.io/projected/7f5aea7c-8d2c-4073-af50-ac30377c7e98-kube-api-access-rwh8h") pod "network-check-target-mr5nf" (UID: "7f5aea7c-8d2c-4073-af50-ac30377c7e98") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 21:02:49.721587 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:49.721247    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:02:49.721587 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:49.721358    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6ctc" podUID="6c684495-6967-47c7-ba35-0b9b37110406"
Jul 20 21:02:49.721587 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:49.721417    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:02:49.721587 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:49.721541    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khqfb" podUID="3f6a3926-e94e-4981-9952-3ec7307a1e48"
Jul 20 21:02:50.720064 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:50.719983    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:02:50.720510 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:50.720107    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr5nf" podUID="7f5aea7c-8d2c-4073-af50-ac30377c7e98"
Jul 20 21:02:51.723270 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:51.723114    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:02:51.723270 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:51.723088    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:02:51.723807 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:51.723299    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6ctc" podUID="6c684495-6967-47c7-ba35-0b9b37110406"
Jul 20 21:02:51.723807 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:51.723661    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khqfb" podUID="3f6a3926-e94e-4981-9952-3ec7307a1e48"
Jul 20 21:02:52.720777 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:52.720734    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:02:52.720946 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:52.720864    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr5nf" podUID="7f5aea7c-8d2c-4073-af50-ac30377c7e98"
Jul 20 21:02:53.720778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:53.720553    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:02:53.721246 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:53.720569    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:02:53.721246 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:53.720885    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6ctc" podUID="6c684495-6967-47c7-ba35-0b9b37110406"
Jul 20 21:02:53.721246 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:53.720958    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khqfb" podUID="3f6a3926-e94e-4981-9952-3ec7307a1e48"
Jul 20 21:02:53.909129 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:53.909041    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf" event={"ID":"c11a809e-92e3-4887-9ca9-c1c3617c3c6c","Type":"ContainerStarted","Data":"ced7c85e729b2c8c67ae30d23ca0f70ae6e42b1eb7614fb536f380007b872495"}
Jul 20 21:02:53.910687 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:53.910656    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzsh6" event={"ID":"1edc053e-71df-4f28-9552-0e7165b736d4","Type":"ContainerStarted","Data":"ad2fe36e3d93eb4e75f2427d111a58aced1f9ba79f7f1cda9f39679d8c88159d"}
Jul 20 21:02:53.912134 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:53.912106    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7k6k7" event={"ID":"bcabbaa9-9e52-49f0-8786-ae54a37210e3","Type":"ContainerStarted","Data":"73d200dbbc54b54fd80eb1170fc006d84214dcb79836dd1c95335126b6aeb32c"}
Jul 20 21:02:53.913585 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:53.913565    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8zfld" event={"ID":"58fafef1-89fd-4094-9993-70ab07fecd85","Type":"ContainerStarted","Data":"c2897282e9af5693bdf163c4b5201f19cd52eff15b58f9e3586832490ad17e0a"}
Jul 20 21:02:53.915240 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:53.915214    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bpvws" event={"ID":"bf582cd6-b53c-45c1-b06b-8eea95f85a95","Type":"ContainerStarted","Data":"1ddd3899f14418dab87d2904c98228586d23d73547896ce296df99ae6ec3dffc"}
Jul 20 21:02:53.916742 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:53.916722    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fhwv7" event={"ID":"06fb01b0-b8c5-45c2-b683-7aeec6af3796","Type":"ContainerStarted","Data":"cd06be777883901e07caad3d2862a188cff82e82d3dc02fa4c8484239e1dc739"}
Jul 20 21:02:53.936744 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:53.936691    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-253.ec2.internal" podStartSLOduration=13.936672681 podStartE2EDuration="13.936672681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 21:02:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:02:44.90293851 +0000 UTC m=+5.806928089" watchObservedRunningTime="2026-07-20 21:02:53.936672681 +0000 UTC m=+14.840662263"
Jul 20 21:02:53.950755 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:53.950710    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8zfld" podStartSLOduration=4.763378073 podStartE2EDuration="14.950691257s" totalImagesPullingTime="10.187313184s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 21:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:02:53.950362533 +0000 UTC m=+14.854352112" watchObservedRunningTime="2026-07-20 21:02:53.950691257 +0000 UTC m=+14.854680836"
Jul 20 21:02:53.984113 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:53.984040    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bpvws" podStartSLOduration=4.797037867 podStartE2EDuration="14.984019408s" totalImagesPullingTime="10.186981541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:02:53.966350994 +0000 UTC m=+14.870340573" watchObservedRunningTime="2026-07-20 21:02:53.984019408 +0000 UTC m=+14.888008989"
Jul 20 21:02:53.984283 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:53.984189    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fhwv7" podStartSLOduration=4.501262311 podStartE2EDuration="14.984181814s" totalImagesPullingTime="10.482919503s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 21:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:02:53.983391902 +0000 UTC m=+14.887381492" watchObservedRunningTime="2026-07-20 21:02:53.984181814 +0000 UTC m=+14.888171391"
Jul 20 21:02:53.998771 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:53.998723    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7k6k7" podStartSLOduration=4.7869086979999995 podStartE2EDuration="14.998709106s" totalImagesPullingTime="10.211800408s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 21:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:02:53.998408235 +0000 UTC m=+14.902397812" watchObservedRunningTime="2026-07-20 21:02:53.998709106 +0000 UTC m=+14.902698684"
Jul 20 21:02:54.720360 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:54.720327    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:02:54.720522 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:54.720434    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr5nf" podUID="7f5aea7c-8d2c-4073-af50-ac30377c7e98"
Jul 20 21:02:54.920705 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:54.920621    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wt9j5" event={"ID":"31d116a1-5993-427a-b6e2-d500f2a38c00","Type":"ContainerStarted","Data":"8b6d421dd8402d6e9357f3a8d39388632925927d5db3f90ef13f2705648b75dc"}
Jul 20 21:02:54.937097 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:54.937023    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wt9j5" podStartSLOduration=5.727082519 podStartE2EDuration="15.937008976s" totalImagesPullingTime="10.209926457s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 21:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:02:54.936479275 +0000 UTC m=+15.840468853" watchObservedRunningTime="2026-07-20 21:02:54.937008976 +0000 UTC m=+15.840998575"
Jul 20 21:02:55.255159 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:55.255057    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bpvws"
Jul 20 21:02:55.720037 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:55.719956    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:02:55.720213 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:55.719951    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:02:55.720213 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:55.720106    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6ctc" podUID="6c684495-6967-47c7-ba35-0b9b37110406"
Jul 20 21:02:55.720213 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:55.720187    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khqfb" podUID="3f6a3926-e94e-4981-9952-3ec7307a1e48"
Jul 20 21:02:55.768499 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:55.768460    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bpvws_bf582cd6-b53c-45c1-b06b-8eea95f85a95/konnectivity-agent/0.log" limit=1024
Jul 20 21:02:55.857793 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:55.857760    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bpvws_bf582cd6-b53c-45c1-b06b-8eea95f85a95/konnectivity-agent/0.log" limit=1024
Jul 20 21:02:55.916154 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:55.916121    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bpvws_bf582cd6-b53c-45c1-b06b-8eea95f85a95/konnectivity-agent/0.log" limit=1024
Jul 20 21:02:56.720484 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:56.720457    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:02:56.721205 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:56.720557    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr5nf" podUID="7f5aea7c-8d2c-4073-af50-ac30377c7e98"
Jul 20 21:02:57.255657 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:57.255614    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs\") pod \"network-metrics-daemon-khqfb\" (UID: \"3f6a3926-e94e-4981-9952-3ec7307a1e48\") " pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:02:57.255846 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:57.255666    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c684495-6967-47c7-ba35-0b9b37110406-original-pull-secret\") pod \"global-pull-secret-syncer-g6ctc\" (UID: \"6c684495-6967-47c7-ba35-0b9b37110406\") " pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:02:57.255846 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:57.255773    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 21:02:57.255846 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:57.255778    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 21:02:57.255846 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:57.255843    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c684495-6967-47c7-ba35-0b9b37110406-original-pull-secret podName:6c684495-6967-47c7-ba35-0b9b37110406 nodeName:}" failed. No retries permitted until 2026-07-20 21:03:13.255823882 +0000 UTC m=+34.159813439 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c684495-6967-47c7-ba35-0b9b37110406-original-pull-secret") pod "global-pull-secret-syncer-g6ctc" (UID: "6c684495-6967-47c7-ba35-0b9b37110406") : object "kube-system"/"original-pull-secret" not registered
Jul 20 21:02:57.256014 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:57.255864    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs podName:3f6a3926-e94e-4981-9952-3ec7307a1e48 nodeName:}" failed. No retries permitted until 2026-07-20 21:03:13.255853656 +0000 UTC m=+34.159843218 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs") pod "network-metrics-daemon-khqfb" (UID: "3f6a3926-e94e-4981-9952-3ec7307a1e48") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 21:02:57.442117 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:57.442086    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bpvws"
Jul 20 21:02:57.442771 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:57.442746    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bpvws"
Jul 20 21:02:57.457621 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:57.457590    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwh8h\" (UniqueName: \"kubernetes.io/projected/7f5aea7c-8d2c-4073-af50-ac30377c7e98-kube-api-access-rwh8h\") pod \"network-check-target-mr5nf\" (UID: \"7f5aea7c-8d2c-4073-af50-ac30377c7e98\") " pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:02:57.457773 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:57.457755    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 21:02:57.457838 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:57.457787    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 21:02:57.457838 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:57.457800    2561 projected.go:196] Error preparing data for projected volume kube-api-access-rwh8h for pod openshift-network-diagnostics/network-check-target-mr5nf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 21:02:57.457919 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:57.457876    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f5aea7c-8d2c-4073-af50-ac30377c7e98-kube-api-access-rwh8h podName:7f5aea7c-8d2c-4073-af50-ac30377c7e98 nodeName:}" failed. No retries permitted until 2026-07-20 21:03:13.457855637 +0000 UTC m=+34.361845213 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rwh8h" (UniqueName: "kubernetes.io/projected/7f5aea7c-8d2c-4073-af50-ac30377c7e98-kube-api-access-rwh8h") pod "network-check-target-mr5nf" (UID: "7f5aea7c-8d2c-4073-af50-ac30377c7e98") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 21:02:57.720625 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:57.720591    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:02:57.721230 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:57.720717    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6ctc" podUID="6c684495-6967-47c7-ba35-0b9b37110406"
Jul 20 21:02:57.721230 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:57.720775    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:02:57.721230 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:57.720920    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khqfb" podUID="3f6a3926-e94e-4981-9952-3ec7307a1e48"
Jul 20 21:02:57.929804 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:57.929711    2561 generic.go:356] "Generic (PLEG): container finished" podID="1edc053e-71df-4f28-9552-0e7165b736d4" containerID="ad2fe36e3d93eb4e75f2427d111a58aced1f9ba79f7f1cda9f39679d8c88159d" exitCode=0
Jul 20 21:02:57.929804 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:57.929784    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzsh6" event={"ID":"1edc053e-71df-4f28-9552-0e7165b736d4","Type":"ContainerDied","Data":"ad2fe36e3d93eb4e75f2427d111a58aced1f9ba79f7f1cda9f39679d8c88159d"}
Jul 20 21:02:57.930601 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:57.930575    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bpvws"
Jul 20 21:02:58.017144 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:58.017054    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bpvws_bf582cd6-b53c-45c1-b06b-8eea95f85a95/konnectivity-agent/0.log" limit=1024
Jul 20 21:02:58.720535 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:58.720497    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:02:58.720714 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:58.720622    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr5nf" podUID="7f5aea7c-8d2c-4073-af50-ac30377c7e98"
Jul 20 21:02:59.721600 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:59.721554    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:02:59.722154 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:02:59.721766    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:02:59.722154 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:59.721873    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6ctc" podUID="6c684495-6967-47c7-ba35-0b9b37110406"
Jul 20 21:02:59.722154 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:02:59.721759    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khqfb" podUID="3f6a3926-e94e-4981-9952-3ec7307a1e48"
Jul 20 21:03:00.720463 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:00.720427    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:03:00.720644 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:00.720549    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr5nf" podUID="7f5aea7c-8d2c-4073-af50-ac30377c7e98"
Jul 20 21:03:01.681253 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:01.681045    2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 20 21:03:01.720741 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:01.720598    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:03:01.720843 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:01.720600    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:03:01.720843 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:01.720833    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khqfb" podUID="3f6a3926-e94e-4981-9952-3ec7307a1e48"
Jul 20 21:03:01.720978 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:01.720943    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6ctc" podUID="6c684495-6967-47c7-ba35-0b9b37110406"
Jul 20 21:03:01.939374 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:01.939340    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4hzwh" event={"ID":"a1111bd9-7008-4387-8e06-83019291010b","Type":"ContainerStarted","Data":"5d7f1536b9aaa4bc0bb52d77efde8f60bc2927fb870cf23b96692ae4153ca3fe"}
Jul 20 21:03:01.941294 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:01.941269    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf" event={"ID":"c11a809e-92e3-4887-9ca9-c1c3617c3c6c","Type":"ContainerStarted","Data":"67dcc938d67151799ab61797ee8544e0ce368cba3d2c433b41bf1332cd863052"}
Jul 20 21:03:01.943957 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:01.943931    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv" event={"ID":"0a812cf7-8e6f-48e2-a22a-29539fa392da","Type":"ContainerStarted","Data":"851d03c5bee6466b4e2dc6097bf346ebe5f6cdbe34a37f03c06792be84e4cfd8"}
Jul 20 21:03:01.944044 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:01.944010    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv" event={"ID":"0a812cf7-8e6f-48e2-a22a-29539fa392da","Type":"ContainerStarted","Data":"ce4980b3388e8ea9fd15fedb8dada0b249d20184cf1b3eaf37958bcbd3eba7d9"}
Jul 20 21:03:01.944119 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:01.944061    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv" event={"ID":"0a812cf7-8e6f-48e2-a22a-29539fa392da","Type":"ContainerStarted","Data":"7886e4344ccfed14f44974bce7701b48e1d5fa9402f202432b671dc34bdf3315"}
Jul 20 21:03:01.944167 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:01.944126    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv" event={"ID":"0a812cf7-8e6f-48e2-a22a-29539fa392da","Type":"ContainerStarted","Data":"0cbd66c527646b2a5c31d60ce38d32bea4f992b6e0569bea7ddbb8a3c75cec83"}
Jul 20 21:03:01.958364 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:01.958321    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4hzwh" podStartSLOduration=4.232390122 podStartE2EDuration="22.958300223s" totalImagesPullingTime="18.725910101s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 21:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:03:01.957715121 +0000 UTC m=+22.861704701" watchObservedRunningTime="2026-07-20 21:03:01.958300223 +0000 UTC m=+22.862289802"
Jul 20 21:03:02.228068 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:02.227956    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bpvws_bf582cd6-b53c-45c1-b06b-8eea95f85a95/konnectivity-agent/0.log" limit=1024
Jul 20 21:03:02.301011 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:02.300981    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bpvws_bf582cd6-b53c-45c1-b06b-8eea95f85a95/konnectivity-agent/0.log" limit=1024
Jul 20 21:03:02.598377 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:02.598211    2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-20T21:03:01.681248696Z","UUID":"775421df-6130-42c8-9a55-74536c60bfa5","Handler":null,"Name":"","Endpoint":""}
Jul 20 21:03:02.600114 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:02.600090    2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 20 21:03:02.600258 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:02.600137    2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 20 21:03:02.721002 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:02.720969    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:03:02.721451 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:02.721122    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr5nf" podUID="7f5aea7c-8d2c-4073-af50-ac30377c7e98"
Jul 20 21:03:02.950520 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:02.950440    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv" event={"ID":"0a812cf7-8e6f-48e2-a22a-29539fa392da","Type":"ContainerStarted","Data":"47859cb0c66f73e87c6ce1bb77166a23d0d00c927b751c8d82b781707ff5fa51"}
Jul 20 21:03:02.950766 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:02.950523    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv" event={"ID":"0a812cf7-8e6f-48e2-a22a-29539fa392da","Type":"ContainerStarted","Data":"3640d08a3abe397cd23db08bec3855a61bc1fb6999dee3deb43f7b34c52f6a86"}
Jul 20 21:03:03.122783 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:03.122753    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bpvws_bf582cd6-b53c-45c1-b06b-8eea95f85a95/konnectivity-agent/0.log" limit=1024
Jul 20 21:03:03.207030 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:03.206951    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bpvws_bf582cd6-b53c-45c1-b06b-8eea95f85a95/konnectivity-agent/0.log" limit=1024
Jul 20 21:03:03.451620 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:03.451582    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bpvws_bf582cd6-b53c-45c1-b06b-8eea95f85a95/konnectivity-agent/0.log" limit=1024
Jul 20 21:03:03.719991 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:03.719960    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:03:03.719991 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:03.719976    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:03:03.720254 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:03.720115    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khqfb" podUID="3f6a3926-e94e-4981-9952-3ec7307a1e48"
Jul 20 21:03:03.720313 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:03.720259    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6ctc" podUID="6c684495-6967-47c7-ba35-0b9b37110406"
Jul 20 21:03:03.954557 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:03.954517    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf" event={"ID":"c11a809e-92e3-4887-9ca9-c1c3617c3c6c","Type":"ContainerStarted","Data":"cd42d9d258bed58e91f953559b8b7b574037462fc134d8040e3f36b3dae1b643"}
Jul 20 21:03:03.973937 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:03.973863    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8k9vf" podStartSLOduration=5.009704133 podStartE2EDuration="24.973848331s" totalImagesPullingTime="19.964144198s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 21:02:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:03:03.973378879 +0000 UTC m=+24.877368456" watchObservedRunningTime="2026-07-20 21:03:03.973848331 +0000 UTC m=+24.877837909"
Jul 20 21:03:04.720748 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:04.720660    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:03:04.720909 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:04.720794    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr5nf" podUID="7f5aea7c-8d2c-4073-af50-ac30377c7e98"
Jul 20 21:03:04.961032 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:04.960991    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv" event={"ID":"0a812cf7-8e6f-48e2-a22a-29539fa392da","Type":"ContainerStarted","Data":"1f569d6fbac2cb0dbfa66505fe658d543ade969e807ecaed302cc6455bed72f3"}
Jul 20 21:03:05.720668 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:05.720629    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:03:05.720838 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:05.720758    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6ctc" podUID="6c684495-6967-47c7-ba35-0b9b37110406"
Jul 20 21:03:05.720911 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:05.720865    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:03:05.721024 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:05.720994    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khqfb" podUID="3f6a3926-e94e-4981-9952-3ec7307a1e48"
Jul 20 21:03:06.720777 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:06.720591    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:03:06.721404 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:06.720858    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr5nf" podUID="7f5aea7c-8d2c-4073-af50-ac30377c7e98"
Jul 20 21:03:06.968038 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:06.967914    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv" event={"ID":"0a812cf7-8e6f-48e2-a22a-29539fa392da","Type":"ContainerStarted","Data":"3866af83643c4c1ab136d95c46a7ae39613f240da17300a09af697562e96341b"}
Jul 20 21:03:06.968229 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:06.968159    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:03:06.968297 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:06.968231    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:03:06.969670 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:06.969643    2561 generic.go:356] "Generic (PLEG): container finished" podID="1edc053e-71df-4f28-9552-0e7165b736d4" containerID="8363bd7fbe38d3786f89cf2172142bc993b6cc0abf781e1878a079c402bbc31e" exitCode=0
Jul 20 21:03:06.969791 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:06.969691    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzsh6" event={"ID":"1edc053e-71df-4f28-9552-0e7165b736d4","Type":"ContainerDied","Data":"8363bd7fbe38d3786f89cf2172142bc993b6cc0abf781e1878a079c402bbc31e"}
Jul 20 21:03:06.984967 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:06.984947    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:03:07.000811 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:07.000767    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv" podStartSLOduration=9.269151824 podStartE2EDuration="28.000753888s" totalImagesPullingTime="18.731602064s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 21:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:03:06.992789069 +0000 UTC m=+27.896778658" watchObservedRunningTime="2026-07-20 21:03:07.000753888 +0000 UTC m=+27.904743466"
Jul 20 21:03:07.721812 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:07.720977    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:03:07.721812 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:07.721132    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6ctc" podUID="6c684495-6967-47c7-ba35-0b9b37110406"
Jul 20 21:03:07.721812 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:07.721602    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:03:07.721812 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:07.721713    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khqfb" podUID="3f6a3926-e94e-4981-9952-3ec7307a1e48"
Jul 20 21:03:07.972755 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:07.972684    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:03:07.986817 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:07.986791    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:03:08.440057 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:08.440017    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-khqfb"]
Jul 20 21:03:08.440253 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:08.440171    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:03:08.440320 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:08.440297    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khqfb" podUID="3f6a3926-e94e-4981-9952-3ec7307a1e48"
Jul 20 21:03:08.440571 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:08.440551    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-g6ctc"]
Jul 20 21:03:08.440698 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:08.440632    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:03:08.440742 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:08.440712    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6ctc" podUID="6c684495-6967-47c7-ba35-0b9b37110406"
Jul 20 21:03:08.442385 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:08.442366    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mr5nf"]
Jul 20 21:03:08.442467 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:08.442442    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:03:08.442526 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:08.442508    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr5nf" podUID="7f5aea7c-8d2c-4073-af50-ac30377c7e98"
Jul 20 21:03:08.977008 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:08.976975    2561 generic.go:356] "Generic (PLEG): container finished" podID="1edc053e-71df-4f28-9552-0e7165b736d4" containerID="30d3fb099aa94d30b5757020876875422ab5d8f35abb4c306c6899b42fd054bc" exitCode=0
Jul 20 21:03:08.977424 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:08.977049    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzsh6" event={"ID":"1edc053e-71df-4f28-9552-0e7165b736d4","Type":"ContainerDied","Data":"30d3fb099aa94d30b5757020876875422ab5d8f35abb4c306c6899b42fd054bc"}
Jul 20 21:03:09.721268 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:09.721018    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:03:09.721433 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:09.721312    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6ctc" podUID="6c684495-6967-47c7-ba35-0b9b37110406"
Jul 20 21:03:10.720691 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:10.720596    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:03:10.721231 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:10.720703    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr5nf" podUID="7f5aea7c-8d2c-4073-af50-ac30377c7e98"
Jul 20 21:03:10.721231 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:10.720766    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:03:10.721231 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:10.720889    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khqfb" podUID="3f6a3926-e94e-4981-9952-3ec7307a1e48"
Jul 20 21:03:10.984481 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:10.984403    2561 generic.go:356] "Generic (PLEG): container finished" podID="1edc053e-71df-4f28-9552-0e7165b736d4" containerID="83a554d0da8ee8024a31bfd561406f0a770f689049ee430c2f6d300412083514" exitCode=0
Jul 20 21:03:10.984481 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:10.984465    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzsh6" event={"ID":"1edc053e-71df-4f28-9552-0e7165b736d4","Type":"ContainerDied","Data":"83a554d0da8ee8024a31bfd561406f0a770f689049ee430c2f6d300412083514"}
Jul 20 21:03:11.720557 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:11.720520    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:03:11.720749 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:11.720659    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6ctc" podUID="6c684495-6967-47c7-ba35-0b9b37110406"
Jul 20 21:03:12.720640 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:12.720606    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:03:12.720808 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:12.720744    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mr5nf" podUID="7f5aea7c-8d2c-4073-af50-ac30377c7e98"
Jul 20 21:03:12.720808 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:12.720789    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:03:12.721247 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:12.720920    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khqfb" podUID="3f6a3926-e94e-4981-9952-3ec7307a1e48"
Jul 20 21:03:13.276206 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:13.276167    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs\") pod \"network-metrics-daemon-khqfb\" (UID: \"3f6a3926-e94e-4981-9952-3ec7307a1e48\") " pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:03:13.276206 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:13.276211    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c684495-6967-47c7-ba35-0b9b37110406-original-pull-secret\") pod \"global-pull-secret-syncer-g6ctc\" (UID: \"6c684495-6967-47c7-ba35-0b9b37110406\") " pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:03:13.276452 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:13.276339    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 21:03:13.276452 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:13.276340    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 21:03:13.276452 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:13.276408    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c684495-6967-47c7-ba35-0b9b37110406-original-pull-secret podName:6c684495-6967-47c7-ba35-0b9b37110406 nodeName:}" failed. No retries permitted until 2026-07-20 21:03:45.276387994 +0000 UTC m=+66.180377555 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c684495-6967-47c7-ba35-0b9b37110406-original-pull-secret") pod "global-pull-secret-syncer-g6ctc" (UID: "6c684495-6967-47c7-ba35-0b9b37110406") : object "kube-system"/"original-pull-secret" not registered
Jul 20 21:03:13.276452 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:13.276429    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs podName:3f6a3926-e94e-4981-9952-3ec7307a1e48 nodeName:}" failed. No retries permitted until 2026-07-20 21:03:45.276417988 +0000 UTC m=+66.180407553 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs") pod "network-metrics-daemon-khqfb" (UID: "3f6a3926-e94e-4981-9952-3ec7307a1e48") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 21:03:13.477761 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:13.477731    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwh8h\" (UniqueName: \"kubernetes.io/projected/7f5aea7c-8d2c-4073-af50-ac30377c7e98-kube-api-access-rwh8h\") pod \"network-check-target-mr5nf\" (UID: \"7f5aea7c-8d2c-4073-af50-ac30377c7e98\") " pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:03:13.477940 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:13.477915    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 21:03:13.478007 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:13.477950    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 21:03:13.478007 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:13.477964    2561 projected.go:196] Error preparing data for projected volume kube-api-access-rwh8h for pod openshift-network-diagnostics/network-check-target-mr5nf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 21:03:13.478096 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:13.478028    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f5aea7c-8d2c-4073-af50-ac30377c7e98-kube-api-access-rwh8h podName:7f5aea7c-8d2c-4073-af50-ac30377c7e98 nodeName:}" failed. No retries permitted until 2026-07-20 21:03:45.478008372 +0000 UTC m=+66.381997940 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rwh8h" (UniqueName: "kubernetes.io/projected/7f5aea7c-8d2c-4073-af50-ac30377c7e98-kube-api-access-rwh8h") pod "network-check-target-mr5nf" (UID: "7f5aea7c-8d2c-4073-af50-ac30377c7e98") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 21:03:13.720610 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:13.720577    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:03:13.720795 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:13.720724    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g6ctc" podUID="6c684495-6967-47c7-ba35-0b9b37110406"
Jul 20 21:03:14.398308 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.398278    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-253.ec2.internal" event="NodeReady"
Jul 20 21:03:14.398844 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.398440    2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 20 21:03:14.450803 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.450766    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-prms6"]
Jul 20 21:03:14.453186 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.453163    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-prms6"
Jul 20 21:03:14.454217 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.454193    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9g6f9"]
Jul 20 21:03:14.456263 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.456243    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9g6f9"
Jul 20 21:03:14.456436 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.456346    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 20 21:03:14.456436 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.456406    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 20 21:03:14.456855 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.456809    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-n29b9\""
Jul 20 21:03:14.459355 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.459335    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 20 21:03:14.459680 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.459662    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 20 21:03:14.459895 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.459876    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 20 21:03:14.460221 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.460161    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2wsgb\""
Jul 20 21:03:14.468700 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.468674    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9g6f9"]
Jul 20 21:03:14.472013 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.471977    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-prms6"]
Jul 20 21:03:14.586893 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.586838    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8e2b83ca-b2c3-4899-92ed-16890097cd04-config-volume\") pod \"dns-default-prms6\" (UID: \"8e2b83ca-b2c3-4899-92ed-16890097cd04\") " pod="openshift-dns/dns-default-prms6"
Jul 20 21:03:14.586893 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.586895    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqpmr\" (UniqueName: \"kubernetes.io/projected/32060f49-8c0d-425a-a22f-8f032ddf9dea-kube-api-access-mqpmr\") pod \"ingress-canary-9g6f9\" (UID: \"32060f49-8c0d-425a-a22f-8f032ddf9dea\") " pod="openshift-ingress-canary/ingress-canary-9g6f9"
Jul 20 21:03:14.587154 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.586918    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e2b83ca-b2c3-4899-92ed-16890097cd04-tmp-dir\") pod \"dns-default-prms6\" (UID: \"8e2b83ca-b2c3-4899-92ed-16890097cd04\") " pod="openshift-dns/dns-default-prms6"
Jul 20 21:03:14.587154 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.586960    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrgtn\" (UniqueName: \"kubernetes.io/projected/8e2b83ca-b2c3-4899-92ed-16890097cd04-kube-api-access-xrgtn\") pod \"dns-default-prms6\" (UID: \"8e2b83ca-b2c3-4899-92ed-16890097cd04\") " pod="openshift-dns/dns-default-prms6"
Jul 20 21:03:14.587154 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.586997    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32060f49-8c0d-425a-a22f-8f032ddf9dea-cert\") pod \"ingress-canary-9g6f9\" (UID: \"32060f49-8c0d-425a-a22f-8f032ddf9dea\") " pod="openshift-ingress-canary/ingress-canary-9g6f9"
Jul 20 21:03:14.587154 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.587023    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls\") pod \"dns-default-prms6\" (UID: \"8e2b83ca-b2c3-4899-92ed-16890097cd04\") " pod="openshift-dns/dns-default-prms6"
Jul 20 21:03:14.688131 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.688033    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrgtn\" (UniqueName: \"kubernetes.io/projected/8e2b83ca-b2c3-4899-92ed-16890097cd04-kube-api-access-xrgtn\") pod \"dns-default-prms6\" (UID: \"8e2b83ca-b2c3-4899-92ed-16890097cd04\") " pod="openshift-dns/dns-default-prms6"
Jul 20 21:03:14.688131 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.688112    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32060f49-8c0d-425a-a22f-8f032ddf9dea-cert\") pod \"ingress-canary-9g6f9\" (UID: \"32060f49-8c0d-425a-a22f-8f032ddf9dea\") " pod="openshift-ingress-canary/ingress-canary-9g6f9"
Jul 20 21:03:14.688356 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.688140    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls\") pod \"dns-default-prms6\" (UID: \"8e2b83ca-b2c3-4899-92ed-16890097cd04\") " pod="openshift-dns/dns-default-prms6"
Jul 20 21:03:14.688356 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.688218    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8e2b83ca-b2c3-4899-92ed-16890097cd04-config-volume\") pod \"dns-default-prms6\" (UID: \"8e2b83ca-b2c3-4899-92ed-16890097cd04\") " pod="openshift-dns/dns-default-prms6"
Jul 20 21:03:14.688356 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:14.688266    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 21:03:14.688356 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.688272    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqpmr\" (UniqueName: \"kubernetes.io/projected/32060f49-8c0d-425a-a22f-8f032ddf9dea-kube-api-access-mqpmr\") pod \"ingress-canary-9g6f9\" (UID: \"32060f49-8c0d-425a-a22f-8f032ddf9dea\") " pod="openshift-ingress-canary/ingress-canary-9g6f9"
Jul 20 21:03:14.688356 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.688300    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e2b83ca-b2c3-4899-92ed-16890097cd04-tmp-dir\") pod \"dns-default-prms6\" (UID: \"8e2b83ca-b2c3-4899-92ed-16890097cd04\") " pod="openshift-dns/dns-default-prms6"
Jul 20 21:03:14.688356 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:14.688336    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls podName:8e2b83ca-b2c3-4899-92ed-16890097cd04 nodeName:}" failed. No retries permitted until 2026-07-20 21:03:15.188317402 +0000 UTC m=+36.092306964 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls") pod "dns-default-prms6" (UID: "8e2b83ca-b2c3-4899-92ed-16890097cd04") : secret "dns-default-metrics-tls" not found
Jul 20 21:03:14.688662 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.688644    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e2b83ca-b2c3-4899-92ed-16890097cd04-tmp-dir\") pod \"dns-default-prms6\" (UID: \"8e2b83ca-b2c3-4899-92ed-16890097cd04\") " pod="openshift-dns/dns-default-prms6"
Jul 20 21:03:14.688915 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.688897    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8e2b83ca-b2c3-4899-92ed-16890097cd04-config-volume\") pod \"dns-default-prms6\" (UID: \"8e2b83ca-b2c3-4899-92ed-16890097cd04\") " pod="openshift-dns/dns-default-prms6"
Jul 20 21:03:14.692801 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.692779    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32060f49-8c0d-425a-a22f-8f032ddf9dea-cert\") pod \"ingress-canary-9g6f9\" (UID: \"32060f49-8c0d-425a-a22f-8f032ddf9dea\") " pod="openshift-ingress-canary/ingress-canary-9g6f9"
Jul 20 21:03:14.696965 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.696944    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrgtn\" (UniqueName: \"kubernetes.io/projected/8e2b83ca-b2c3-4899-92ed-16890097cd04-kube-api-access-xrgtn\") pod \"dns-default-prms6\" (UID: \"8e2b83ca-b2c3-4899-92ed-16890097cd04\") " pod="openshift-dns/dns-default-prms6"
Jul 20 21:03:14.697779 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.697747    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqpmr\" (UniqueName: \"kubernetes.io/projected/32060f49-8c0d-425a-a22f-8f032ddf9dea-kube-api-access-mqpmr\") pod \"ingress-canary-9g6f9\" (UID: \"32060f49-8c0d-425a-a22f-8f032ddf9dea\") " pod="openshift-ingress-canary/ingress-canary-9g6f9"
Jul 20 21:03:14.720634 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.720595    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:03:14.720997 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.720789    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:03:14.724374 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.724109    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 20 21:03:14.724374 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.724127    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 20 21:03:14.724374 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.724159    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 20 21:03:14.724374 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.724270    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hpjcj\""
Jul 20 21:03:14.724374 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.724270    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lfgsb\""
Jul 20 21:03:14.772720 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:14.772687    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9g6f9"
Jul 20 21:03:15.191556 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:15.191520    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls\") pod \"dns-default-prms6\" (UID: \"8e2b83ca-b2c3-4899-92ed-16890097cd04\") " pod="openshift-dns/dns-default-prms6"
Jul 20 21:03:15.192018 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:15.191986    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 21:03:15.192173 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:15.192093    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls podName:8e2b83ca-b2c3-4899-92ed-16890097cd04 nodeName:}" failed. No retries permitted until 2026-07-20 21:03:16.192053616 +0000 UTC m=+37.096043186 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls") pod "dns-default-prms6" (UID: "8e2b83ca-b2c3-4899-92ed-16890097cd04") : secret "dns-default-metrics-tls" not found
Jul 20 21:03:15.723857 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:15.723293    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:03:15.728328 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:15.727851    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 20 21:03:15.791612 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:15.791588    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9g6f9"]
Jul 20 21:03:15.794814 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:03:15.794785    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32060f49_8c0d_425a_a22f_8f032ddf9dea.slice/crio-3f61236ab54f52d6242c5401edfcb449a4ceb0f6958226a56bcd795be42e5026 WatchSource:0}: Error finding container 3f61236ab54f52d6242c5401edfcb449a4ceb0f6958226a56bcd795be42e5026: Status 404 returned error can't find the container with id 3f61236ab54f52d6242c5401edfcb449a4ceb0f6958226a56bcd795be42e5026
Jul 20 21:03:15.998428 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:15.998323    2561 generic.go:356] "Generic (PLEG): container finished" podID="1edc053e-71df-4f28-9552-0e7165b736d4" containerID="fa6bff78037663368154546c3f9acd5e91286531a821dcc4a745be38ebf9c492" exitCode=0
Jul 20 21:03:15.998428 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:15.998394    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzsh6" event={"ID":"1edc053e-71df-4f28-9552-0e7165b736d4","Type":"ContainerDied","Data":"fa6bff78037663368154546c3f9acd5e91286531a821dcc4a745be38ebf9c492"}
Jul 20 21:03:15.999471 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:15.999443    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9g6f9" event={"ID":"32060f49-8c0d-425a-a22f-8f032ddf9dea","Type":"ContainerStarted","Data":"3f61236ab54f52d6242c5401edfcb449a4ceb0f6958226a56bcd795be42e5026"}
Jul 20 21:03:16.199539 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:16.199495    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls\") pod \"dns-default-prms6\" (UID: \"8e2b83ca-b2c3-4899-92ed-16890097cd04\") " pod="openshift-dns/dns-default-prms6"
Jul 20 21:03:16.199681 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:16.199640    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 21:03:16.199779 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:16.199702    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls podName:8e2b83ca-b2c3-4899-92ed-16890097cd04 nodeName:}" failed. No retries permitted until 2026-07-20 21:03:18.199685206 +0000 UTC m=+39.103674762 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls") pod "dns-default-prms6" (UID: "8e2b83ca-b2c3-4899-92ed-16890097cd04") : secret "dns-default-metrics-tls" not found
Jul 20 21:03:17.005574 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:17.005542    2561 generic.go:356] "Generic (PLEG): container finished" podID="1edc053e-71df-4f28-9552-0e7165b736d4" containerID="20d558f9872793dcd929ccbed0b4dce736c33eb48932a9655b02096668494ccb" exitCode=0
Jul 20 21:03:17.005574 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:17.005583    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzsh6" event={"ID":"1edc053e-71df-4f28-9552-0e7165b736d4","Type":"ContainerDied","Data":"20d558f9872793dcd929ccbed0b4dce736c33eb48932a9655b02096668494ccb"}
Jul 20 21:03:18.011879 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:18.011685    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzsh6" event={"ID":"1edc053e-71df-4f28-9552-0e7165b736d4","Type":"ContainerStarted","Data":"b0fa390016b4015a32ba74a83c41701dc1a070aa81700d67ad7260df70f8bb61"}
Jul 20 21:03:18.212732 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:18.212690    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls\") pod \"dns-default-prms6\" (UID: \"8e2b83ca-b2c3-4899-92ed-16890097cd04\") " pod="openshift-dns/dns-default-prms6"
Jul 20 21:03:18.212907 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:18.212864    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 21:03:18.212951 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:18.212929    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls podName:8e2b83ca-b2c3-4899-92ed-16890097cd04 nodeName:}" failed. No retries permitted until 2026-07-20 21:03:22.212911691 +0000 UTC m=+43.116901263 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls") pod "dns-default-prms6" (UID: "8e2b83ca-b2c3-4899-92ed-16890097cd04") : secret "dns-default-metrics-tls" not found
Jul 20 21:03:20.017279 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:20.017239    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9g6f9" event={"ID":"32060f49-8c0d-425a-a22f-8f032ddf9dea","Type":"ContainerStarted","Data":"e62e53f683f183882b460f5a1b7ef1562f0bab91b4f8facd31d6a4e766417090"}
Jul 20 21:03:20.038051 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:20.038001    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rzsh6" podStartSLOduration=15.546483972 podStartE2EDuration="41.03797821s" totalImagesPullingTime="25.491494238s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 21:02:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:03:18.053957691 +0000 UTC m=+38.957947269" watchObservedRunningTime="2026-07-20 21:03:20.03797821 +0000 UTC m=+40.941967770"
Jul 20 21:03:20.038370 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:20.038345    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9g6f9" podStartSLOduration=2.864032683 podStartE2EDuration="6.038336441s" totalImagesPullingTime="3.174303758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:03:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:03:20.037228261 +0000 UTC m=+40.941217840" watchObservedRunningTime="2026-07-20 21:03:20.038336441 +0000 UTC m=+40.942326033"
Jul 20 21:03:22.241043 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:22.241005    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls\") pod \"dns-default-prms6\" (UID: \"8e2b83ca-b2c3-4899-92ed-16890097cd04\") " pod="openshift-dns/dns-default-prms6"
Jul 20 21:03:22.241465 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:22.241176    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 21:03:22.241465 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:22.241243    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls podName:8e2b83ca-b2c3-4899-92ed-16890097cd04 nodeName:}" failed. No retries permitted until 2026-07-20 21:03:30.241228101 +0000 UTC m=+51.145217662 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls") pod "dns-default-prms6" (UID: "8e2b83ca-b2c3-4899-92ed-16890097cd04") : secret "dns-default-metrics-tls" not found
Jul 20 21:03:30.301916 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:30.301877    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls\") pod \"dns-default-prms6\" (UID: \"8e2b83ca-b2c3-4899-92ed-16890097cd04\") " pod="openshift-dns/dns-default-prms6"
Jul 20 21:03:30.302520 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:30.302039    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 21:03:30.302520 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:30.302127    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls podName:8e2b83ca-b2c3-4899-92ed-16890097cd04 nodeName:}" failed. No retries permitted until 2026-07-20 21:03:46.302107964 +0000 UTC m=+67.206097521 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls") pod "dns-default-prms6" (UID: "8e2b83ca-b2c3-4899-92ed-16890097cd04") : secret "dns-default-metrics-tls" not found
Jul 20 21:03:39.999588 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:39.999562    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w6bqv"
Jul 20 21:03:45.298861 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:45.298805    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs\") pod \"network-metrics-daemon-khqfb\" (UID: \"3f6a3926-e94e-4981-9952-3ec7307a1e48\") " pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:03:45.298861 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:45.298864    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c684495-6967-47c7-ba35-0b9b37110406-original-pull-secret\") pod \"global-pull-secret-syncer-g6ctc\" (UID: \"6c684495-6967-47c7-ba35-0b9b37110406\") " pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:03:45.301742 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:45.301721    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 20 21:03:45.301801 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:45.301769    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 20 21:03:45.309830 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:45.309806    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 20 21:03:45.309933 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:45.309866    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs podName:3f6a3926-e94e-4981-9952-3ec7307a1e48 nodeName:}" failed. No retries permitted until 2026-07-20 21:04:49.309850923 +0000 UTC m=+130.213840478 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs") pod "network-metrics-daemon-khqfb" (UID: "3f6a3926-e94e-4981-9952-3ec7307a1e48") : secret "metrics-daemon-secret" not found
Jul 20 21:03:45.313533 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:45.313506    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c684495-6967-47c7-ba35-0b9b37110406-original-pull-secret\") pod \"global-pull-secret-syncer-g6ctc\" (UID: \"6c684495-6967-47c7-ba35-0b9b37110406\") " pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:03:45.437461 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:45.437424    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g6ctc"
Jul 20 21:03:45.500219 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:45.500186    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwh8h\" (UniqueName: \"kubernetes.io/projected/7f5aea7c-8d2c-4073-af50-ac30377c7e98-kube-api-access-rwh8h\") pod \"network-check-target-mr5nf\" (UID: \"7f5aea7c-8d2c-4073-af50-ac30377c7e98\") " pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:03:45.503343 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:45.503318    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 20 21:03:45.512913 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:45.512888    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 20 21:03:45.523327 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:45.523303    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwh8h\" (UniqueName: \"kubernetes.io/projected/7f5aea7c-8d2c-4073-af50-ac30377c7e98-kube-api-access-rwh8h\") pod \"network-check-target-mr5nf\" (UID: \"7f5aea7c-8d2c-4073-af50-ac30377c7e98\") " pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:03:45.556729 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:45.556635    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-g6ctc"]
Jul 20 21:03:45.560307 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:03:45.560279    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c684495_6967_47c7_ba35_0b9b37110406.slice/crio-792928f323142ccbe88ae71e2e978b4ead0ca475599012a2954654e15b3ab439 WatchSource:0}: Error finding container 792928f323142ccbe88ae71e2e978b4ead0ca475599012a2954654e15b3ab439: Status 404 returned error can't find the container with id 792928f323142ccbe88ae71e2e978b4ead0ca475599012a2954654e15b3ab439
Jul 20 21:03:45.635447 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:45.635414    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lfgsb\""
Jul 20 21:03:45.642665 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:45.642642    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:03:45.762990 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:45.762931    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mr5nf"]
Jul 20 21:03:45.766611 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:03:45.766575    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f5aea7c_8d2c_4073_af50_ac30377c7e98.slice/crio-a215e1a42b419c34570a23ba948e278effbca179b343b9ad1e379aa33bda6aed WatchSource:0}: Error finding container a215e1a42b419c34570a23ba948e278effbca179b343b9ad1e379aa33bda6aed: Status 404 returned error can't find the container with id a215e1a42b419c34570a23ba948e278effbca179b343b9ad1e379aa33bda6aed
Jul 20 21:03:46.080240 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:46.080205    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mr5nf" event={"ID":"7f5aea7c-8d2c-4073-af50-ac30377c7e98","Type":"ContainerStarted","Data":"a215e1a42b419c34570a23ba948e278effbca179b343b9ad1e379aa33bda6aed"}
Jul 20 21:03:46.081295 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:46.081274    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-g6ctc" event={"ID":"6c684495-6967-47c7-ba35-0b9b37110406","Type":"ContainerStarted","Data":"792928f323142ccbe88ae71e2e978b4ead0ca475599012a2954654e15b3ab439"}
Jul 20 21:03:46.306735 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:46.306698    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls\") pod \"dns-default-prms6\" (UID: \"8e2b83ca-b2c3-4899-92ed-16890097cd04\") " pod="openshift-dns/dns-default-prms6"
Jul 20 21:03:46.307277 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:46.306838    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 21:03:46.307277 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:03:46.306906    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls podName:8e2b83ca-b2c3-4899-92ed-16890097cd04 nodeName:}" failed. No retries permitted until 2026-07-20 21:04:18.306891347 +0000 UTC m=+99.210880916 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls") pod "dns-default-prms6" (UID: "8e2b83ca-b2c3-4899-92ed-16890097cd04") : secret "dns-default-metrics-tls" not found
Jul 20 21:03:51.095757 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:51.095722    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mr5nf" event={"ID":"7f5aea7c-8d2c-4073-af50-ac30377c7e98","Type":"ContainerStarted","Data":"1281253da9a8b25ada59a7dd9aa44c88aa4f54e8feb675fa2fcd74f51a7052dd"}
Jul 20 21:03:51.096305 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:51.095807    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:03:51.097135 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:51.097109    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-g6ctc" event={"ID":"6c684495-6967-47c7-ba35-0b9b37110406","Type":"ContainerStarted","Data":"5a8067d0ba343847ee4d570b4e3777c24a3e58c2b913ddff120b73e3b21822b6"}
Jul 20 21:03:51.112227 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:51.112182    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mr5nf" podStartSLOduration=67.552455887 podStartE2EDuration="1m12.112168052s" totalImagesPullingTime="4.559712165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:03:51.111441193 +0000 UTC m=+72.015430769" watchObservedRunningTime="2026-07-20 21:03:51.112168052 +0000 UTC m=+72.016157628"
Jul 20 21:03:51.126033 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:03:51.125981    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-g6ctc" podStartSLOduration=67.356338086 podStartE2EDuration="1m12.125971083s" totalImagesPullingTime="4.769632997s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 21:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:03:51.125182826 +0000 UTC m=+72.029172403" watchObservedRunningTime="2026-07-20 21:03:51.125971083 +0000 UTC m=+72.029960661"
Jul 20 21:04:18.331700 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:04:18.331657    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls\") pod \"dns-default-prms6\" (UID: \"8e2b83ca-b2c3-4899-92ed-16890097cd04\") " pod="openshift-dns/dns-default-prms6"
Jul 20 21:04:18.332120 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:04:18.331771    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 21:04:18.332120 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:04:18.331824    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls podName:8e2b83ca-b2c3-4899-92ed-16890097cd04 nodeName:}" failed. No retries permitted until 2026-07-20 21:05:22.331809942 +0000 UTC m=+163.235799497 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls") pod "dns-default-prms6" (UID: "8e2b83ca-b2c3-4899-92ed-16890097cd04") : secret "dns-default-metrics-tls" not found
Jul 20 21:04:22.102663 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:04:22.102633    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mr5nf"
Jul 20 21:04:49.336012 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:04:49.335956    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs\") pod \"network-metrics-daemon-khqfb\" (UID: \"3f6a3926-e94e-4981-9952-3ec7307a1e48\") " pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:04:49.336501 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:04:49.336131    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 20 21:04:49.336501 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:04:49.336210    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs podName:3f6a3926-e94e-4981-9952-3ec7307a1e48 nodeName:}" failed. No retries permitted until 2026-07-20 21:06:51.336193202 +0000 UTC m=+252.240182758 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs") pod "network-metrics-daemon-khqfb" (UID: "3f6a3926-e94e-4981-9952-3ec7307a1e48") : secret "metrics-daemon-secret" not found
Jul 20 21:05:17.466284 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:05:17.466212    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-prms6" podUID="8e2b83ca-b2c3-4899-92ed-16890097cd04"
Jul 20 21:05:17.738613 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:05:17.738525    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-khqfb" podUID="3f6a3926-e94e-4981-9952-3ec7307a1e48"
Jul 20 21:05:18.295882 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:05:18.295847    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-prms6"
Jul 20 21:05:22.370941 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:05:22.370896    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls\") pod \"dns-default-prms6\" (UID: \"8e2b83ca-b2c3-4899-92ed-16890097cd04\") " pod="openshift-dns/dns-default-prms6"
Jul 20 21:05:22.371367 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:05:22.371021    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 21:05:22.371367 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:05:22.371103    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls podName:8e2b83ca-b2c3-4899-92ed-16890097cd04 nodeName:}" failed. No retries permitted until 2026-07-20 21:07:24.371067109 +0000 UTC m=+285.275056669 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls") pod "dns-default-prms6" (UID: "8e2b83ca-b2c3-4899-92ed-16890097cd04") : secret "dns-default-metrics-tls" not found
Jul 20 21:05:31.721140 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:05:31.721027    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:06:50.965349 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:50.965314    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-ckxwr"]
Jul 20 21:06:50.967722 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:50.967700    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-ckxwr"
Jul 20 21:06:50.976424 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:50.976400    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 20 21:06:50.976530 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:50.976414    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 20 21:06:50.977749 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:50.977731    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 20 21:06:50.978938 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:50.978920    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 20 21:06:50.979058 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:50.978962    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-52xjs\""
Jul 20 21:06:50.982158 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:50.982142    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 20 21:06:51.034246 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.034218    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-ckxwr"]
Jul 20 21:06:51.034879 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.034859    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-vm2s6"]
Jul 20 21:06:51.036598 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.036582    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-vm2s6"
Jul 20 21:06:51.037927 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.037904    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-64d7f6dfcb-zmq82"]
Jul 20 21:06:51.039701 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.039685    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:06:51.040345 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.040325    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-5dtx9\""
Jul 20 21:06:51.045688 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.045672    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 20 21:06:51.045956 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.045943    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 20 21:06:51.048468 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.048451    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 20 21:06:51.048659 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.048647    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xv4xs\""
Jul 20 21:06:51.050108 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.050093    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 20 21:06:51.052365 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.052351    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 20 21:06:51.052423 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.052384    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 20 21:06:51.052638 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.052620    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxzr6\" (UniqueName: \"kubernetes.io/projected/844a855d-9faf-4af8-96f4-1985596720ea-kube-api-access-jxzr6\") pod \"insights-operator-555d9d7885-ckxwr\" (UID: \"844a855d-9faf-4af8-96f4-1985596720ea\") " pod="openshift-insights/insights-operator-555d9d7885-ckxwr"
Jul 20 21:06:51.052682 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.052647    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/844a855d-9faf-4af8-96f4-1985596720ea-tmp\") pod \"insights-operator-555d9d7885-ckxwr\" (UID: \"844a855d-9faf-4af8-96f4-1985596720ea\") " pod="openshift-insights/insights-operator-555d9d7885-ckxwr"
Jul 20 21:06:51.052682 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.052665    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/844a855d-9faf-4af8-96f4-1985596720ea-snapshots\") pod \"insights-operator-555d9d7885-ckxwr\" (UID: \"844a855d-9faf-4af8-96f4-1985596720ea\") " pod="openshift-insights/insights-operator-555d9d7885-ckxwr"
Jul 20 21:06:51.052757 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.052689    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/844a855d-9faf-4af8-96f4-1985596720ea-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-ckxwr\" (UID: \"844a855d-9faf-4af8-96f4-1985596720ea\") " pod="openshift-insights/insights-operator-555d9d7885-ckxwr"
Jul 20 21:06:51.052757 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.052749    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/844a855d-9faf-4af8-96f4-1985596720ea-serving-cert\") pod \"insights-operator-555d9d7885-ckxwr\" (UID: \"844a855d-9faf-4af8-96f4-1985596720ea\") " pod="openshift-insights/insights-operator-555d9d7885-ckxwr"
Jul 20 21:06:51.052851 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.052803    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/844a855d-9faf-4af8-96f4-1985596720ea-service-ca-bundle\") pod \"insights-operator-555d9d7885-ckxwr\" (UID: \"844a855d-9faf-4af8-96f4-1985596720ea\") " pod="openshift-insights/insights-operator-555d9d7885-ckxwr"
Jul 20 21:06:51.069973 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.069927    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-64d7f6dfcb-zmq82"]
Jul 20 21:06:51.073897 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.073875    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-vm2s6"]
Jul 20 21:06:51.119043 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.119009    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-p84ql"]
Jul 20 21:06:51.124494 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.124466    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-p84ql"
Jul 20 21:06:51.128690 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.128670    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-fz7g7"]
Jul 20 21:06:51.130370 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.130340    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-fz7g7"
Jul 20 21:06:51.132553 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.132536    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 20 21:06:51.132657 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.132577    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 20 21:06:51.132657 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.132594    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-trb82\""
Jul 20 21:06:51.132862 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.132847    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 20 21:06:51.132934 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.132874    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 20 21:06:51.138320 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.138304    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 20 21:06:51.138391 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.138382    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 20 21:06:51.138507 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.138492    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 20 21:06:51.143323 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.143306    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-bv2bm\""
Jul 20 21:06:51.144000 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.143979    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-p84ql"]
Jul 20 21:06:51.146901 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.146884    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 20 21:06:51.153286 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.153262    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/844a855d-9faf-4af8-96f4-1985596720ea-tmp\") pod \"insights-operator-555d9d7885-ckxwr\" (UID: \"844a855d-9faf-4af8-96f4-1985596720ea\") " pod="openshift-insights/insights-operator-555d9d7885-ckxwr"
Jul 20 21:06:51.153376 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.153295    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3fb0dff-5b39-431b-959b-ba6636770fe4-service-ca-bundle\") pod \"router-default-64d7f6dfcb-zmq82\" (UID: \"e3fb0dff-5b39-431b-959b-ba6636770fe4\") " pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:06:51.153376 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.153313    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e3fb0dff-5b39-431b-959b-ba6636770fe4-stats-auth\") pod \"router-default-64d7f6dfcb-zmq82\" (UID: \"e3fb0dff-5b39-431b-959b-ba6636770fe4\") " pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:06:51.153376 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.153331    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46r67\" (UniqueName: \"kubernetes.io/projected/9b1daf89-91d9-45b4-81d0-a7c8aa3edd2b-kube-api-access-46r67\") pod \"network-check-source-7bc5b6bf85-vm2s6\" (UID: \"9b1daf89-91d9-45b4-81d0-a7c8aa3edd2b\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-vm2s6"
Jul 20 21:06:51.153515 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.153419    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/844a855d-9faf-4af8-96f4-1985596720ea-snapshots\") pod \"insights-operator-555d9d7885-ckxwr\" (UID: \"844a855d-9faf-4af8-96f4-1985596720ea\") " pod="openshift-insights/insights-operator-555d9d7885-ckxwr"
Jul 20 21:06:51.153515 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.153464    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e3fb0dff-5b39-431b-959b-ba6636770fe4-default-certificate\") pod \"router-default-64d7f6dfcb-zmq82\" (UID: \"e3fb0dff-5b39-431b-959b-ba6636770fe4\") " pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:06:51.153615 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.153515    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/844a855d-9faf-4af8-96f4-1985596720ea-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-ckxwr\" (UID: \"844a855d-9faf-4af8-96f4-1985596720ea\") " pod="openshift-insights/insights-operator-555d9d7885-ckxwr"
Jul 20 21:06:51.153710 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.153644    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/844a855d-9faf-4af8-96f4-1985596720ea-serving-cert\") pod \"insights-operator-555d9d7885-ckxwr\" (UID: \"844a855d-9faf-4af8-96f4-1985596720ea\") " pod="openshift-insights/insights-operator-555d9d7885-ckxwr"
Jul 20 21:06:51.153710 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.153684    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/844a855d-9faf-4af8-96f4-1985596720ea-tmp\") pod \"insights-operator-555d9d7885-ckxwr\" (UID: \"844a855d-9faf-4af8-96f4-1985596720ea\") " pod="openshift-insights/insights-operator-555d9d7885-ckxwr"
Jul 20 21:06:51.153817 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.153682    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/844a855d-9faf-4af8-96f4-1985596720ea-service-ca-bundle\") pod \"insights-operator-555d9d7885-ckxwr\" (UID: \"844a855d-9faf-4af8-96f4-1985596720ea\") " pod="openshift-insights/insights-operator-555d9d7885-ckxwr"
Jul 20 21:06:51.153817 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.153778    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3fb0dff-5b39-431b-959b-ba6636770fe4-metrics-certs\") pod \"router-default-64d7f6dfcb-zmq82\" (UID: \"e3fb0dff-5b39-431b-959b-ba6636770fe4\") " pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:06:51.153917 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.153818    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxzr6\" (UniqueName: \"kubernetes.io/projected/844a855d-9faf-4af8-96f4-1985596720ea-kube-api-access-jxzr6\") pod \"insights-operator-555d9d7885-ckxwr\" (UID: \"844a855d-9faf-4af8-96f4-1985596720ea\") " pod="openshift-insights/insights-operator-555d9d7885-ckxwr"
Jul 20 21:06:51.153917 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.153844    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-747rv\" (UniqueName: \"kubernetes.io/projected/e3fb0dff-5b39-431b-959b-ba6636770fe4-kube-api-access-747rv\") pod \"router-default-64d7f6dfcb-zmq82\" (UID: \"e3fb0dff-5b39-431b-959b-ba6636770fe4\") " pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:06:51.154155 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.154128    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/844a855d-9faf-4af8-96f4-1985596720ea-snapshots\") pod \"insights-operator-555d9d7885-ckxwr\" (UID: \"844a855d-9faf-4af8-96f4-1985596720ea\") " pod="openshift-insights/insights-operator-555d9d7885-ckxwr"
Jul 20 21:06:51.154515 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.154498    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/844a855d-9faf-4af8-96f4-1985596720ea-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-ckxwr\" (UID: \"844a855d-9faf-4af8-96f4-1985596720ea\") " pod="openshift-insights/insights-operator-555d9d7885-ckxwr"
Jul 20 21:06:51.154858 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.154836    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/844a855d-9faf-4af8-96f4-1985596720ea-service-ca-bundle\") pod \"insights-operator-555d9d7885-ckxwr\" (UID: \"844a855d-9faf-4af8-96f4-1985596720ea\") " pod="openshift-insights/insights-operator-555d9d7885-ckxwr"
Jul 20 21:06:51.155374 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.155355    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 20 21:06:51.156119 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.156098    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-fz7g7"]
Jul 20 21:06:51.156253 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.156234    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/844a855d-9faf-4af8-96f4-1985596720ea-serving-cert\") pod \"insights-operator-555d9d7885-ckxwr\" (UID: \"844a855d-9faf-4af8-96f4-1985596720ea\") " pod="openshift-insights/insights-operator-555d9d7885-ckxwr"
Jul 20 21:06:51.173984 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.173960    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxzr6\" (UniqueName: \"kubernetes.io/projected/844a855d-9faf-4af8-96f4-1985596720ea-kube-api-access-jxzr6\") pod \"insights-operator-555d9d7885-ckxwr\" (UID: \"844a855d-9faf-4af8-96f4-1985596720ea\") " pod="openshift-insights/insights-operator-555d9d7885-ckxwr"
Jul 20 21:06:51.254849 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.254759    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e3fb0dff-5b39-431b-959b-ba6636770fe4-default-certificate\") pod \"router-default-64d7f6dfcb-zmq82\" (UID: \"e3fb0dff-5b39-431b-959b-ba6636770fe4\") " pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:06:51.254849 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.254795    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng92q\" (UniqueName: \"kubernetes.io/projected/a6f93445-e1d9-4ef1-bbb4-aba4bd103cd6-kube-api-access-ng92q\") pod \"service-ca-operator-5bcbc94c78-p84ql\" (UID: \"a6f93445-e1d9-4ef1-bbb4-aba4bd103cd6\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-p84ql"
Jul 20 21:06:51.254849 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.254829    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f3754f5e-54ec-4d60-9e1f-9290b4a82d07-config\") pod \"console-operator-565b7b569d-fz7g7\" (UID: \"f3754f5e-54ec-4d60-9e1f-9290b4a82d07\") " pod="openshift-console-operator/console-operator-565b7b569d-fz7g7"
Jul 20 21:06:51.255165 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.254870    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a6f93445-e1d9-4ef1-bbb4-aba4bd103cd6-serving-cert\") pod \"service-ca-operator-5bcbc94c78-p84ql\" (UID: \"a6f93445-e1d9-4ef1-bbb4-aba4bd103cd6\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-p84ql"
Jul 20 21:06:51.255165 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.254887    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a6f93445-e1d9-4ef1-bbb4-aba4bd103cd6-config\") pod \"service-ca-operator-5bcbc94c78-p84ql\" (UID: \"a6f93445-e1d9-4ef1-bbb4-aba4bd103cd6\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-p84ql"
Jul 20 21:06:51.255165 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.254912    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3fb0dff-5b39-431b-959b-ba6636770fe4-metrics-certs\") pod \"router-default-64d7f6dfcb-zmq82\" (UID: \"e3fb0dff-5b39-431b-959b-ba6636770fe4\") " pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:06:51.255165 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.254927    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f3754f5e-54ec-4d60-9e1f-9290b4a82d07-trusted-ca\") pod \"console-operator-565b7b569d-fz7g7\" (UID: \"f3754f5e-54ec-4d60-9e1f-9290b4a82d07\") " pod="openshift-console-operator/console-operator-565b7b569d-fz7g7"
Jul 20 21:06:51.255165 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.254955    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-747rv\" (UniqueName: \"kubernetes.io/projected/e3fb0dff-5b39-431b-959b-ba6636770fe4-kube-api-access-747rv\") pod \"router-default-64d7f6dfcb-zmq82\" (UID: \"e3fb0dff-5b39-431b-959b-ba6636770fe4\") " pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:06:51.255165 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.254973    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3fb0dff-5b39-431b-959b-ba6636770fe4-service-ca-bundle\") pod \"router-default-64d7f6dfcb-zmq82\" (UID: \"e3fb0dff-5b39-431b-959b-ba6636770fe4\") " pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:06:51.255165 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.254987    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e3fb0dff-5b39-431b-959b-ba6636770fe4-stats-auth\") pod \"router-default-64d7f6dfcb-zmq82\" (UID: \"e3fb0dff-5b39-431b-959b-ba6636770fe4\") " pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:06:51.255165 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:51.254991    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 20 21:06:51.255165 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.255002    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46r67\" (UniqueName: \"kubernetes.io/projected/9b1daf89-91d9-45b4-81d0-a7c8aa3edd2b-kube-api-access-46r67\") pod \"network-check-source-7bc5b6bf85-vm2s6\" (UID: \"9b1daf89-91d9-45b4-81d0-a7c8aa3edd2b\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-vm2s6"
Jul 20 21:06:51.255165 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.255020    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f3754f5e-54ec-4d60-9e1f-9290b4a82d07-serving-cert\") pod \"console-operator-565b7b569d-fz7g7\" (UID: \"f3754f5e-54ec-4d60-9e1f-9290b4a82d07\") " pod="openshift-console-operator/console-operator-565b7b569d-fz7g7"
Jul 20 21:06:51.255165 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:51.255061    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3fb0dff-5b39-431b-959b-ba6636770fe4-metrics-certs podName:e3fb0dff-5b39-431b-959b-ba6636770fe4 nodeName:}" failed. No retries permitted until 2026-07-20 21:06:51.755040559 +0000 UTC m=+252.659030115 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3fb0dff-5b39-431b-959b-ba6636770fe4-metrics-certs") pod "router-default-64d7f6dfcb-zmq82" (UID: "e3fb0dff-5b39-431b-959b-ba6636770fe4") : secret "router-metrics-certs-default" not found
Jul 20 21:06:51.255165 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:51.255133    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e3fb0dff-5b39-431b-959b-ba6636770fe4-service-ca-bundle podName:e3fb0dff-5b39-431b-959b-ba6636770fe4 nodeName:}" failed. No retries permitted until 2026-07-20 21:06:51.75511565 +0000 UTC m=+252.659105205 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e3fb0dff-5b39-431b-959b-ba6636770fe4-service-ca-bundle") pod "router-default-64d7f6dfcb-zmq82" (UID: "e3fb0dff-5b39-431b-959b-ba6636770fe4") : configmap references non-existent config key: service-ca.crt
Jul 20 21:06:51.255580 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.255162    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpznb\" (UniqueName: \"kubernetes.io/projected/f3754f5e-54ec-4d60-9e1f-9290b4a82d07-kube-api-access-cpznb\") pod \"console-operator-565b7b569d-fz7g7\" (UID: \"f3754f5e-54ec-4d60-9e1f-9290b4a82d07\") " pod="openshift-console-operator/console-operator-565b7b569d-fz7g7"
Jul 20 21:06:51.257277 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.257256    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e3fb0dff-5b39-431b-959b-ba6636770fe4-default-certificate\") pod \"router-default-64d7f6dfcb-zmq82\" (UID: \"e3fb0dff-5b39-431b-959b-ba6636770fe4\") " pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:06:51.257367 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.257321    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e3fb0dff-5b39-431b-959b-ba6636770fe4-stats-auth\") pod \"router-default-64d7f6dfcb-zmq82\" (UID: \"e3fb0dff-5b39-431b-959b-ba6636770fe4\") " pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:06:51.264940 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.264915    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46r67\" (UniqueName: \"kubernetes.io/projected/9b1daf89-91d9-45b4-81d0-a7c8aa3edd2b-kube-api-access-46r67\") pod \"network-check-source-7bc5b6bf85-vm2s6\" (UID: \"9b1daf89-91d9-45b4-81d0-a7c8aa3edd2b\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-vm2s6"
Jul 20 21:06:51.275068 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.275048    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-747rv\" (UniqueName: \"kubernetes.io/projected/e3fb0dff-5b39-431b-959b-ba6636770fe4-kube-api-access-747rv\") pod \"router-default-64d7f6dfcb-zmq82\" (UID: \"e3fb0dff-5b39-431b-959b-ba6636770fe4\") " pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:06:51.275831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.275814    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-ckxwr"
Jul 20 21:06:51.346563 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.346534    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-vm2s6"
Jul 20 21:06:51.355642 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.355618    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f3754f5e-54ec-4d60-9e1f-9290b4a82d07-serving-cert\") pod \"console-operator-565b7b569d-fz7g7\" (UID: \"f3754f5e-54ec-4d60-9e1f-9290b4a82d07\") " pod="openshift-console-operator/console-operator-565b7b569d-fz7g7"
Jul 20 21:06:51.355785 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.355651    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpznb\" (UniqueName: \"kubernetes.io/projected/f3754f5e-54ec-4d60-9e1f-9290b4a82d07-kube-api-access-cpznb\") pod \"console-operator-565b7b569d-fz7g7\" (UID: \"f3754f5e-54ec-4d60-9e1f-9290b4a82d07\") " pod="openshift-console-operator/console-operator-565b7b569d-fz7g7"
Jul 20 21:06:51.355785 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.355679    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng92q\" (UniqueName: \"kubernetes.io/projected/a6f93445-e1d9-4ef1-bbb4-aba4bd103cd6-kube-api-access-ng92q\") pod \"service-ca-operator-5bcbc94c78-p84ql\" (UID: \"a6f93445-e1d9-4ef1-bbb4-aba4bd103cd6\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-p84ql"
Jul 20 21:06:51.355785 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.355712    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f3754f5e-54ec-4d60-9e1f-9290b4a82d07-config\") pod \"console-operator-565b7b569d-fz7g7\" (UID: \"f3754f5e-54ec-4d60-9e1f-9290b4a82d07\") " pod="openshift-console-operator/console-operator-565b7b569d-fz7g7"
Jul 20 21:06:51.355785 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.355749    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs\") pod \"network-metrics-daemon-khqfb\" (UID: \"3f6a3926-e94e-4981-9952-3ec7307a1e48\") " pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:06:51.355785 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.355774    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a6f93445-e1d9-4ef1-bbb4-aba4bd103cd6-serving-cert\") pod \"service-ca-operator-5bcbc94c78-p84ql\" (UID: \"a6f93445-e1d9-4ef1-bbb4-aba4bd103cd6\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-p84ql"
Jul 20 21:06:51.357292 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.355963    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a6f93445-e1d9-4ef1-bbb4-aba4bd103cd6-config\") pod \"service-ca-operator-5bcbc94c78-p84ql\" (UID: \"a6f93445-e1d9-4ef1-bbb4-aba4bd103cd6\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-p84ql"
Jul 20 21:06:51.357292 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:51.355996    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 20 21:06:51.357292 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.356036    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f3754f5e-54ec-4d60-9e1f-9290b4a82d07-trusted-ca\") pod \"console-operator-565b7b569d-fz7g7\" (UID: \"f3754f5e-54ec-4d60-9e1f-9290b4a82d07\") " pod="openshift-console-operator/console-operator-565b7b569d-fz7g7"
Jul 20 21:06:51.357292 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:51.356084    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs podName:3f6a3926-e94e-4981-9952-3ec7307a1e48 nodeName:}" failed. No retries permitted until 2026-07-20 21:08:53.356050756 +0000 UTC m=+374.260040331 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs") pod "network-metrics-daemon-khqfb" (UID: "3f6a3926-e94e-4981-9952-3ec7307a1e48") : secret "metrics-daemon-secret" not found
Jul 20 21:06:51.357292 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.356715    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a6f93445-e1d9-4ef1-bbb4-aba4bd103cd6-config\") pod \"service-ca-operator-5bcbc94c78-p84ql\" (UID: \"a6f93445-e1d9-4ef1-bbb4-aba4bd103cd6\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-p84ql"
Jul 20 21:06:51.357292 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.356874    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f3754f5e-54ec-4d60-9e1f-9290b4a82d07-config\") pod \"console-operator-565b7b569d-fz7g7\" (UID: \"f3754f5e-54ec-4d60-9e1f-9290b4a82d07\") " pod="openshift-console-operator/console-operator-565b7b569d-fz7g7"
Jul 20 21:06:51.357292 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.357215    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f3754f5e-54ec-4d60-9e1f-9290b4a82d07-trusted-ca\") pod \"console-operator-565b7b569d-fz7g7\" (UID: \"f3754f5e-54ec-4d60-9e1f-9290b4a82d07\") " pod="openshift-console-operator/console-operator-565b7b569d-fz7g7"
Jul 20 21:06:51.358247 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.358222    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a6f93445-e1d9-4ef1-bbb4-aba4bd103cd6-serving-cert\") pod \"service-ca-operator-5bcbc94c78-p84ql\" (UID: \"a6f93445-e1d9-4ef1-bbb4-aba4bd103cd6\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-p84ql"
Jul 20 21:06:51.358396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.358375    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f3754f5e-54ec-4d60-9e1f-9290b4a82d07-serving-cert\") pod \"console-operator-565b7b569d-fz7g7\" (UID: \"f3754f5e-54ec-4d60-9e1f-9290b4a82d07\") " pod="openshift-console-operator/console-operator-565b7b569d-fz7g7"
Jul 20 21:06:51.364429 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.364391    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng92q\" (UniqueName: \"kubernetes.io/projected/a6f93445-e1d9-4ef1-bbb4-aba4bd103cd6-kube-api-access-ng92q\") pod \"service-ca-operator-5bcbc94c78-p84ql\" (UID: \"a6f93445-e1d9-4ef1-bbb4-aba4bd103cd6\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-p84ql"
Jul 20 21:06:51.367172 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.367151    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpznb\" (UniqueName: \"kubernetes.io/projected/f3754f5e-54ec-4d60-9e1f-9290b4a82d07-kube-api-access-cpznb\") pod \"console-operator-565b7b569d-fz7g7\" (UID: \"f3754f5e-54ec-4d60-9e1f-9290b4a82d07\") " pod="openshift-console-operator/console-operator-565b7b569d-fz7g7"
Jul 20 21:06:51.413318 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.413282    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-ckxwr"]
Jul 20 21:06:51.419008 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:06:51.418979    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod844a855d_9faf_4af8_96f4_1985596720ea.slice/crio-c4ab0736f614b62bc1924aa2ac0814f3ca488ac1e58e723f8cf056f240e48a31 WatchSource:0}: Error finding container c4ab0736f614b62bc1924aa2ac0814f3ca488ac1e58e723f8cf056f240e48a31: Status 404 returned error can't find the container with id c4ab0736f614b62bc1924aa2ac0814f3ca488ac1e58e723f8cf056f240e48a31
Jul 20 21:06:51.433423 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.433393    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-p84ql"
Jul 20 21:06:51.440147 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.440065    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-fz7g7"
Jul 20 21:06:51.479469 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.479437    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-vm2s6"]
Jul 20 21:06:51.484605 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:06:51.484518    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b1daf89_91d9_45b4_81d0_a7c8aa3edd2b.slice/crio-7c30b1827349b7b7f94837fc7179cff28269dea6f567157872980c9ca34920f1 WatchSource:0}: Error finding container 7c30b1827349b7b7f94837fc7179cff28269dea6f567157872980c9ca34920f1: Status 404 returned error can't find the container with id 7c30b1827349b7b7f94837fc7179cff28269dea6f567157872980c9ca34920f1
Jul 20 21:06:51.508659 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.508578    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-vm2s6" event={"ID":"9b1daf89-91d9-45b4-81d0-a7c8aa3edd2b","Type":"ContainerStarted","Data":"7c30b1827349b7b7f94837fc7179cff28269dea6f567157872980c9ca34920f1"}
Jul 20 21:06:51.509823 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.509798    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-ckxwr" event={"ID":"844a855d-9faf-4af8-96f4-1985596720ea","Type":"ContainerStarted","Data":"c4ab0736f614b62bc1924aa2ac0814f3ca488ac1e58e723f8cf056f240e48a31"}
Jul 20 21:06:51.578388 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.578314    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-p84ql"]
Jul 20 21:06:51.583845 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:06:51.583817    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6f93445_e1d9_4ef1_bbb4_aba4bd103cd6.slice/crio-3cf2624c1e61157aeaad26c320476273968532d9cdb2c845611b8b0e78662b9e WatchSource:0}: Error finding container 3cf2624c1e61157aeaad26c320476273968532d9cdb2c845611b8b0e78662b9e: Status 404 returned error can't find the container with id 3cf2624c1e61157aeaad26c320476273968532d9cdb2c845611b8b0e78662b9e
Jul 20 21:06:51.597264 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.597228    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-fz7g7"]
Jul 20 21:06:51.601380 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:06:51.601353    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3754f5e_54ec_4d60_9e1f_9290b4a82d07.slice/crio-18794583f50739f6e86a45955b164543233a761856ea09042e6ab3a30830a327 WatchSource:0}: Error finding container 18794583f50739f6e86a45955b164543233a761856ea09042e6ab3a30830a327: Status 404 returned error can't find the container with id 18794583f50739f6e86a45955b164543233a761856ea09042e6ab3a30830a327
Jul 20 21:06:51.760087 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.759985    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3fb0dff-5b39-431b-959b-ba6636770fe4-metrics-certs\") pod \"router-default-64d7f6dfcb-zmq82\" (UID: \"e3fb0dff-5b39-431b-959b-ba6636770fe4\") " pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:06:51.760087 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:51.760026    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3fb0dff-5b39-431b-959b-ba6636770fe4-service-ca-bundle\") pod \"router-default-64d7f6dfcb-zmq82\" (UID: \"e3fb0dff-5b39-431b-959b-ba6636770fe4\") " pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:06:51.760300 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:51.760156    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 20 21:06:51.760300 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:51.760169    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e3fb0dff-5b39-431b-959b-ba6636770fe4-service-ca-bundle podName:e3fb0dff-5b39-431b-959b-ba6636770fe4 nodeName:}" failed. No retries permitted until 2026-07-20 21:06:52.760155277 +0000 UTC m=+253.664144833 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e3fb0dff-5b39-431b-959b-ba6636770fe4-service-ca-bundle") pod "router-default-64d7f6dfcb-zmq82" (UID: "e3fb0dff-5b39-431b-959b-ba6636770fe4") : configmap references non-existent config key: service-ca.crt
Jul 20 21:06:51.760300 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:51.760218    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3fb0dff-5b39-431b-959b-ba6636770fe4-metrics-certs podName:e3fb0dff-5b39-431b-959b-ba6636770fe4 nodeName:}" failed. No retries permitted until 2026-07-20 21:06:52.76020463 +0000 UTC m=+253.664194186 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3fb0dff-5b39-431b-959b-ba6636770fe4-metrics-certs") pod "router-default-64d7f6dfcb-zmq82" (UID: "e3fb0dff-5b39-431b-959b-ba6636770fe4") : secret "router-metrics-certs-default" not found
Jul 20 21:06:52.514622 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:52.514548    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-fz7g7" event={"ID":"f3754f5e-54ec-4d60-9e1f-9290b4a82d07","Type":"ContainerStarted","Data":"18794583f50739f6e86a45955b164543233a761856ea09042e6ab3a30830a327"}
Jul 20 21:06:52.516039 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:52.516005    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-p84ql" event={"ID":"a6f93445-e1d9-4ef1-bbb4-aba4bd103cd6","Type":"ContainerStarted","Data":"3cf2624c1e61157aeaad26c320476273968532d9cdb2c845611b8b0e78662b9e"}
Jul 20 21:06:52.518316 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:52.518290    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-vm2s6" event={"ID":"9b1daf89-91d9-45b4-81d0-a7c8aa3edd2b","Type":"ContainerStarted","Data":"49d9fa2185454199912d6cb4e291743258725cfd4a3a00a048e3a2cc9ccf4796"}
Jul 20 21:06:52.548294 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:52.548228    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-vm2s6" podStartSLOduration=1.548204588 podStartE2EDuration="1.548204588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:06:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:06:52.546713225 +0000 UTC m=+253.450702802" watchObservedRunningTime="2026-07-20 21:06:52.548204588 +0000 UTC m=+253.452194167"
Jul 20 21:06:52.770063 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:52.769977    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3fb0dff-5b39-431b-959b-ba6636770fe4-service-ca-bundle\") pod \"router-default-64d7f6dfcb-zmq82\" (UID: \"e3fb0dff-5b39-431b-959b-ba6636770fe4\") " pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:06:52.770259 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:52.770144    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3fb0dff-5b39-431b-959b-ba6636770fe4-metrics-certs\") pod \"router-default-64d7f6dfcb-zmq82\" (UID: \"e3fb0dff-5b39-431b-959b-ba6636770fe4\") " pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:06:52.770259 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:52.770212    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e3fb0dff-5b39-431b-959b-ba6636770fe4-service-ca-bundle podName:e3fb0dff-5b39-431b-959b-ba6636770fe4 nodeName:}" failed. No retries permitted until 2026-07-20 21:06:54.770189243 +0000 UTC m=+255.674178816 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e3fb0dff-5b39-431b-959b-ba6636770fe4-service-ca-bundle") pod "router-default-64d7f6dfcb-zmq82" (UID: "e3fb0dff-5b39-431b-959b-ba6636770fe4") : configmap references non-existent config key: service-ca.crt
Jul 20 21:06:52.770373 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:52.770261    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 20 21:06:52.770373 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:52.770310    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3fb0dff-5b39-431b-959b-ba6636770fe4-metrics-certs podName:e3fb0dff-5b39-431b-959b-ba6636770fe4 nodeName:}" failed. No retries permitted until 2026-07-20 21:06:54.770297479 +0000 UTC m=+255.674287036 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3fb0dff-5b39-431b-959b-ba6636770fe4-metrics-certs") pod "router-default-64d7f6dfcb-zmq82" (UID: "e3fb0dff-5b39-431b-959b-ba6636770fe4") : secret "router-metrics-certs-default" not found
Jul 20 21:06:54.787704 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:54.787672    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3fb0dff-5b39-431b-959b-ba6636770fe4-metrics-certs\") pod \"router-default-64d7f6dfcb-zmq82\" (UID: \"e3fb0dff-5b39-431b-959b-ba6636770fe4\") " pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:06:54.788184 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:54.787736    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3fb0dff-5b39-431b-959b-ba6636770fe4-service-ca-bundle\") pod \"router-default-64d7f6dfcb-zmq82\" (UID: \"e3fb0dff-5b39-431b-959b-ba6636770fe4\") " pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:06:54.788184 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:54.787836    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 20 21:06:54.788184 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:54.787918    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3fb0dff-5b39-431b-959b-ba6636770fe4-metrics-certs podName:e3fb0dff-5b39-431b-959b-ba6636770fe4 nodeName:}" failed. No retries permitted until 2026-07-20 21:06:58.787896977 +0000 UTC m=+259.691886533 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3fb0dff-5b39-431b-959b-ba6636770fe4-metrics-certs") pod "router-default-64d7f6dfcb-zmq82" (UID: "e3fb0dff-5b39-431b-959b-ba6636770fe4") : secret "router-metrics-certs-default" not found
Jul 20 21:06:54.788184 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:54.787937    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e3fb0dff-5b39-431b-959b-ba6636770fe4-service-ca-bundle podName:e3fb0dff-5b39-431b-959b-ba6636770fe4 nodeName:}" failed. No retries permitted until 2026-07-20 21:06:58.787928847 +0000 UTC m=+259.691918406 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e3fb0dff-5b39-431b-959b-ba6636770fe4-service-ca-bundle") pod "router-default-64d7f6dfcb-zmq82" (UID: "e3fb0dff-5b39-431b-959b-ba6636770fe4") : configmap references non-existent config key: service-ca.crt
Jul 20 21:06:55.529597 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.529562    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/0.log"
Jul 20 21:06:55.529788 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.529608    2561 generic.go:356] "Generic (PLEG): container finished" podID="f3754f5e-54ec-4d60-9e1f-9290b4a82d07" containerID="7eeacefec6996e944dfd5b0e172240463e24073e970e1106ba30aacbff77c58b" exitCode=255
Jul 20 21:06:55.529788 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.529690    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-fz7g7" event={"ID":"f3754f5e-54ec-4d60-9e1f-9290b4a82d07","Type":"ContainerDied","Data":"7eeacefec6996e944dfd5b0e172240463e24073e970e1106ba30aacbff77c58b"}
Jul 20 21:06:55.529937 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.529900    2561 scope.go:117] "RemoveContainer" containerID="7eeacefec6996e944dfd5b0e172240463e24073e970e1106ba30aacbff77c58b"
Jul 20 21:06:55.531124 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.531098    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-p84ql" event={"ID":"a6f93445-e1d9-4ef1-bbb4-aba4bd103cd6","Type":"ContainerStarted","Data":"410741b47af4bff6e6d138d14a744375914009a77227bcf7ca80cbd3c71dbe17"}
Jul 20 21:06:55.532500 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.532475    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-ckxwr" event={"ID":"844a855d-9faf-4af8-96f4-1985596720ea","Type":"ContainerStarted","Data":"99ab6e93590694d606b381ab6408c73b23041e3cea762b9edca89021b053fc5b"}
Jul 20 21:06:55.661993 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.661963    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hpljx"]
Jul 20 21:06:55.664287 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.664266    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:06:55.694947 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.694910    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 20 21:06:55.695120 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.694950    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kgm4r\""
Jul 20 21:06:55.695243 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.695226    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 20 21:06:55.701785 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.701727    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-p84ql" podStartSLOduration=1.500420436 podStartE2EDuration="4.701708169s" totalImagesPullingTime="3.201287733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:06:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:06:55.700909074 +0000 UTC m=+256.604898656" watchObservedRunningTime="2026-07-20 21:06:55.701708169 +0000 UTC m=+256.605697748"
Jul 20 21:06:55.707041 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.707013    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hpljx"]
Jul 20 21:06:55.731246 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.731199    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-ckxwr" podStartSLOduration=2.372333197 podStartE2EDuration="5.731178541s" totalImagesPullingTime="3.358845344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:06:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:06:55.728264052 +0000 UTC m=+256.632253630" watchObservedRunningTime="2026-07-20 21:06:55.731178541 +0000 UTC m=+256.635168119"
Jul 20 21:06:55.798047 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.798021    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7215e114-f9ed-40cb-9de5-c48cef3995cd-data-volume\") pod \"insights-runtime-extractor-hpljx\" (UID: \"7215e114-f9ed-40cb-9de5-c48cef3995cd\") " pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:06:55.798372 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.798092    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7215e114-f9ed-40cb-9de5-c48cef3995cd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hpljx\" (UID: \"7215e114-f9ed-40cb-9de5-c48cef3995cd\") " pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:06:55.798372 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.798174    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7215e114-f9ed-40cb-9de5-c48cef3995cd-crio-socket\") pod \"insights-runtime-extractor-hpljx\" (UID: \"7215e114-f9ed-40cb-9de5-c48cef3995cd\") " pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:06:55.798372 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.798227    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxxnq\" (UniqueName: \"kubernetes.io/projected/7215e114-f9ed-40cb-9de5-c48cef3995cd-kube-api-access-rxxnq\") pod \"insights-runtime-extractor-hpljx\" (UID: \"7215e114-f9ed-40cb-9de5-c48cef3995cd\") " pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:06:55.798372 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.798331    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7215e114-f9ed-40cb-9de5-c48cef3995cd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hpljx\" (UID: \"7215e114-f9ed-40cb-9de5-c48cef3995cd\") " pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:06:55.898758 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.898707    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7215e114-f9ed-40cb-9de5-c48cef3995cd-data-volume\") pod \"insights-runtime-extractor-hpljx\" (UID: \"7215e114-f9ed-40cb-9de5-c48cef3995cd\") " pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:06:55.898758 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.898764    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7215e114-f9ed-40cb-9de5-c48cef3995cd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hpljx\" (UID: \"7215e114-f9ed-40cb-9de5-c48cef3995cd\") " pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:06:55.898980 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.898790    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7215e114-f9ed-40cb-9de5-c48cef3995cd-crio-socket\") pod \"insights-runtime-extractor-hpljx\" (UID: \"7215e114-f9ed-40cb-9de5-c48cef3995cd\") " pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:06:55.898980 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.898807    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxxnq\" (UniqueName: \"kubernetes.io/projected/7215e114-f9ed-40cb-9de5-c48cef3995cd-kube-api-access-rxxnq\") pod \"insights-runtime-extractor-hpljx\" (UID: \"7215e114-f9ed-40cb-9de5-c48cef3995cd\") " pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:06:55.898980 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.898824    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7215e114-f9ed-40cb-9de5-c48cef3995cd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hpljx\" (UID: \"7215e114-f9ed-40cb-9de5-c48cef3995cd\") " pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:06:55.898980 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.898912    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7215e114-f9ed-40cb-9de5-c48cef3995cd-crio-socket\") pod \"insights-runtime-extractor-hpljx\" (UID: \"7215e114-f9ed-40cb-9de5-c48cef3995cd\") " pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:06:55.898980 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:55.898926    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 21:06:55.898980 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:55.898953    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 21:06:55.899198 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:55.898999    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7215e114-f9ed-40cb-9de5-c48cef3995cd-kube-rbac-proxy-cm podName:7215e114-f9ed-40cb-9de5-c48cef3995cd nodeName:}" failed. No retries permitted until 2026-07-20 21:06:56.398978921 +0000 UTC m=+257.302968480 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7215e114-f9ed-40cb-9de5-c48cef3995cd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hpljx" (UID: "7215e114-f9ed-40cb-9de5-c48cef3995cd") : configmap "kube-rbac-proxy" not found
Jul 20 21:06:55.899198 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:55.899012    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7215e114-f9ed-40cb-9de5-c48cef3995cd-insights-runtime-extractor-tls podName:7215e114-f9ed-40cb-9de5-c48cef3995cd nodeName:}" failed. No retries permitted until 2026-07-20 21:06:56.399006675 +0000 UTC m=+257.302996230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7215e114-f9ed-40cb-9de5-c48cef3995cd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hpljx" (UID: "7215e114-f9ed-40cb-9de5-c48cef3995cd") : secret "insights-runtime-extractor-tls" not found
Jul 20 21:06:55.899198 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.899107    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7215e114-f9ed-40cb-9de5-c48cef3995cd-data-volume\") pod \"insights-runtime-extractor-hpljx\" (UID: \"7215e114-f9ed-40cb-9de5-c48cef3995cd\") " pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:06:55.913037 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:55.913005    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxxnq\" (UniqueName: \"kubernetes.io/projected/7215e114-f9ed-40cb-9de5-c48cef3995cd-kube-api-access-rxxnq\") pod \"insights-runtime-extractor-hpljx\" (UID: \"7215e114-f9ed-40cb-9de5-c48cef3995cd\") " pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:06:56.402107 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:56.402046    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7215e114-f9ed-40cb-9de5-c48cef3995cd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hpljx\" (UID: \"7215e114-f9ed-40cb-9de5-c48cef3995cd\") " pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:06:56.402107 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:56.402114    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7215e114-f9ed-40cb-9de5-c48cef3995cd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hpljx\" (UID: \"7215e114-f9ed-40cb-9de5-c48cef3995cd\") " pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:06:56.402404 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:56.402196    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 21:06:56.402404 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:56.402220    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 21:06:56.402404 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:56.402261    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7215e114-f9ed-40cb-9de5-c48cef3995cd-insights-runtime-extractor-tls podName:7215e114-f9ed-40cb-9de5-c48cef3995cd nodeName:}" failed. No retries permitted until 2026-07-20 21:06:57.402246107 +0000 UTC m=+258.306235662 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7215e114-f9ed-40cb-9de5-c48cef3995cd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hpljx" (UID: "7215e114-f9ed-40cb-9de5-c48cef3995cd") : secret "insights-runtime-extractor-tls" not found
Jul 20 21:06:56.402404 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:56.402274    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7215e114-f9ed-40cb-9de5-c48cef3995cd-kube-rbac-proxy-cm podName:7215e114-f9ed-40cb-9de5-c48cef3995cd nodeName:}" failed. No retries permitted until 2026-07-20 21:06:57.402268409 +0000 UTC m=+258.306257965 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7215e114-f9ed-40cb-9de5-c48cef3995cd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hpljx" (UID: "7215e114-f9ed-40cb-9de5-c48cef3995cd") : configmap "kube-rbac-proxy" not found
Jul 20 21:06:56.536615 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:56.536586    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/1.log"
Jul 20 21:06:56.537006 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:56.536992    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/0.log"
Jul 20 21:06:56.537049 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:56.537027    2561 generic.go:356] "Generic (PLEG): container finished" podID="f3754f5e-54ec-4d60-9e1f-9290b4a82d07" containerID="6271f4cd851fa35163db08bed72b801819ddbb1f43e30c9dcbb5ea2c45d81aa1" exitCode=255
Jul 20 21:06:56.537129 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:56.537109    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-fz7g7" event={"ID":"f3754f5e-54ec-4d60-9e1f-9290b4a82d07","Type":"ContainerDied","Data":"6271f4cd851fa35163db08bed72b801819ddbb1f43e30c9dcbb5ea2c45d81aa1"}
Jul 20 21:06:56.537181 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:56.537168    2561 scope.go:117] "RemoveContainer" containerID="7eeacefec6996e944dfd5b0e172240463e24073e970e1106ba30aacbff77c58b"
Jul 20 21:06:56.537392 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:56.537372    2561 scope.go:117] "RemoveContainer" containerID="6271f4cd851fa35163db08bed72b801819ddbb1f43e30c9dcbb5ea2c45d81aa1"
Jul 20 21:06:56.537713 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:56.537688    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-fz7g7_openshift-console-operator(f3754f5e-54ec-4d60-9e1f-9290b4a82d07)\"" pod="openshift-console-operator/console-operator-565b7b569d-fz7g7" podUID="f3754f5e-54ec-4d60-9e1f-9290b4a82d07"
Jul 20 21:06:57.411619 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:57.411577    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7215e114-f9ed-40cb-9de5-c48cef3995cd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hpljx\" (UID: \"7215e114-f9ed-40cb-9de5-c48cef3995cd\") " pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:06:57.411619 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:57.411623    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7215e114-f9ed-40cb-9de5-c48cef3995cd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hpljx\" (UID: \"7215e114-f9ed-40cb-9de5-c48cef3995cd\") " pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:06:57.412144 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:57.411692    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 21:06:57.412144 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:57.411734    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 21:06:57.412144 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:57.411743    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7215e114-f9ed-40cb-9de5-c48cef3995cd-kube-rbac-proxy-cm podName:7215e114-f9ed-40cb-9de5-c48cef3995cd nodeName:}" failed. No retries permitted until 2026-07-20 21:06:59.411730568 +0000 UTC m=+260.315720124 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7215e114-f9ed-40cb-9de5-c48cef3995cd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hpljx" (UID: "7215e114-f9ed-40cb-9de5-c48cef3995cd") : configmap "kube-rbac-proxy" not found
Jul 20 21:06:57.412144 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:57.411817    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7215e114-f9ed-40cb-9de5-c48cef3995cd-insights-runtime-extractor-tls podName:7215e114-f9ed-40cb-9de5-c48cef3995cd nodeName:}" failed. No retries permitted until 2026-07-20 21:06:59.411798763 +0000 UTC m=+260.315788319 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7215e114-f9ed-40cb-9de5-c48cef3995cd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hpljx" (UID: "7215e114-f9ed-40cb-9de5-c48cef3995cd") : secret "insights-runtime-extractor-tls" not found
Jul 20 21:06:57.542173 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:57.542145    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/1.log"
Jul 20 21:06:57.542475 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:57.542459    2561 scope.go:117] "RemoveContainer" containerID="6271f4cd851fa35163db08bed72b801819ddbb1f43e30c9dcbb5ea2c45d81aa1"
Jul 20 21:06:57.542624 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:57.542605    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-fz7g7_openshift-console-operator(f3754f5e-54ec-4d60-9e1f-9290b4a82d07)\"" pod="openshift-console-operator/console-operator-565b7b569d-fz7g7" podUID="f3754f5e-54ec-4d60-9e1f-9290b4a82d07"
Jul 20 21:06:58.230751 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:58.230723    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7k6k7_bcabbaa9-9e52-49f0-8786-ae54a37210e3/dns-node-resolver/0.log"
Jul 20 21:06:58.302407 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:58.302376    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-gvjfj"]
Jul 20 21:06:58.305010 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:58.304989    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-gvjfj"
Jul 20 21:06:58.307735 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:58.307707    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 20 21:06:58.307883 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:58.307862    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 20 21:06:58.308745 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:58.308730    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-5kdf4\""
Jul 20 21:06:58.308867 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:58.308731    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 20 21:06:58.308867 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:58.308765    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 20 21:06:58.316945 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:58.316927    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-gvjfj"]
Jul 20 21:06:58.419604 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:58.419568    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/79b8b7d5-2511-4da3-afe1-3a0c861f22ba-signing-cabundle\") pod \"service-ca-58598695f7-gvjfj\" (UID: \"79b8b7d5-2511-4da3-afe1-3a0c861f22ba\") " pod="openshift-service-ca/service-ca-58598695f7-gvjfj"
Jul 20 21:06:58.420029 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:58.419644    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-456vx\" (UniqueName: \"kubernetes.io/projected/79b8b7d5-2511-4da3-afe1-3a0c861f22ba-kube-api-access-456vx\") pod \"service-ca-58598695f7-gvjfj\" (UID: \"79b8b7d5-2511-4da3-afe1-3a0c861f22ba\") " pod="openshift-service-ca/service-ca-58598695f7-gvjfj"
Jul 20 21:06:58.420029 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:58.419717    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/79b8b7d5-2511-4da3-afe1-3a0c861f22ba-signing-key\") pod \"service-ca-58598695f7-gvjfj\" (UID: \"79b8b7d5-2511-4da3-afe1-3a0c861f22ba\") " pod="openshift-service-ca/service-ca-58598695f7-gvjfj"
Jul 20 21:06:58.520474 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:58.520389    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/79b8b7d5-2511-4da3-afe1-3a0c861f22ba-signing-cabundle\") pod \"service-ca-58598695f7-gvjfj\" (UID: \"79b8b7d5-2511-4da3-afe1-3a0c861f22ba\") " pod="openshift-service-ca/service-ca-58598695f7-gvjfj"
Jul 20 21:06:58.520474 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:58.520447    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-456vx\" (UniqueName: \"kubernetes.io/projected/79b8b7d5-2511-4da3-afe1-3a0c861f22ba-kube-api-access-456vx\") pod \"service-ca-58598695f7-gvjfj\" (UID: \"79b8b7d5-2511-4da3-afe1-3a0c861f22ba\") " pod="openshift-service-ca/service-ca-58598695f7-gvjfj"
Jul 20 21:06:58.520697 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:58.520486    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/79b8b7d5-2511-4da3-afe1-3a0c861f22ba-signing-key\") pod \"service-ca-58598695f7-gvjfj\" (UID: \"79b8b7d5-2511-4da3-afe1-3a0c861f22ba\") " pod="openshift-service-ca/service-ca-58598695f7-gvjfj"
Jul 20 21:06:58.521044 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:58.521024    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/79b8b7d5-2511-4da3-afe1-3a0c861f22ba-signing-cabundle\") pod \"service-ca-58598695f7-gvjfj\" (UID: \"79b8b7d5-2511-4da3-afe1-3a0c861f22ba\") " pod="openshift-service-ca/service-ca-58598695f7-gvjfj"
Jul 20 21:06:58.522790 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:58.522772    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/79b8b7d5-2511-4da3-afe1-3a0c861f22ba-signing-key\") pod \"service-ca-58598695f7-gvjfj\" (UID: \"79b8b7d5-2511-4da3-afe1-3a0c861f22ba\") " pod="openshift-service-ca/service-ca-58598695f7-gvjfj"
Jul 20 21:06:58.532091 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:58.532044    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-456vx\" (UniqueName: \"kubernetes.io/projected/79b8b7d5-2511-4da3-afe1-3a0c861f22ba-kube-api-access-456vx\") pod \"service-ca-58598695f7-gvjfj\" (UID: \"79b8b7d5-2511-4da3-afe1-3a0c861f22ba\") " pod="openshift-service-ca/service-ca-58598695f7-gvjfj"
Jul 20 21:06:58.614090 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:58.614037    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-gvjfj"
Jul 20 21:06:58.739577 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:58.739547    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-gvjfj"]
Jul 20 21:06:58.742441 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:06:58.742411    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79b8b7d5_2511_4da3_afe1_3a0c861f22ba.slice/crio-ad72b7a00b162ef43503be1439813c099833e60701f6823b8c37e150d24d3418 WatchSource:0}: Error finding container ad72b7a00b162ef43503be1439813c099833e60701f6823b8c37e150d24d3418: Status 404 returned error can't find the container with id ad72b7a00b162ef43503be1439813c099833e60701f6823b8c37e150d24d3418
Jul 20 21:06:58.823130 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:58.823094    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3fb0dff-5b39-431b-959b-ba6636770fe4-metrics-certs\") pod \"router-default-64d7f6dfcb-zmq82\" (UID: \"e3fb0dff-5b39-431b-959b-ba6636770fe4\") " pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:06:58.823130 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:58.823143    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3fb0dff-5b39-431b-959b-ba6636770fe4-service-ca-bundle\") pod \"router-default-64d7f6dfcb-zmq82\" (UID: \"e3fb0dff-5b39-431b-959b-ba6636770fe4\") " pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:06:58.823353 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:58.823264    2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 20 21:06:58.823353 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:58.823290    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e3fb0dff-5b39-431b-959b-ba6636770fe4-service-ca-bundle podName:e3fb0dff-5b39-431b-959b-ba6636770fe4 nodeName:}" failed. No retries permitted until 2026-07-20 21:07:06.823275948 +0000 UTC m=+267.727265507 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e3fb0dff-5b39-431b-959b-ba6636770fe4-service-ca-bundle") pod "router-default-64d7f6dfcb-zmq82" (UID: "e3fb0dff-5b39-431b-959b-ba6636770fe4") : configmap references non-existent config key: service-ca.crt
Jul 20 21:06:58.823353 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:58.823320    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3fb0dff-5b39-431b-959b-ba6636770fe4-metrics-certs podName:e3fb0dff-5b39-431b-959b-ba6636770fe4 nodeName:}" failed. No retries permitted until 2026-07-20 21:07:06.823305672 +0000 UTC m=+267.727295228 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3fb0dff-5b39-431b-959b-ba6636770fe4-metrics-certs") pod "router-default-64d7f6dfcb-zmq82" (UID: "e3fb0dff-5b39-431b-959b-ba6636770fe4") : secret "router-metrics-certs-default" not found
Jul 20 21:06:59.232957 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:59.232929    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8zfld_58fafef1-89fd-4094-9993-70ab07fecd85/node-ca/0.log"
Jul 20 21:06:59.426359 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:59.426322    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7215e114-f9ed-40cb-9de5-c48cef3995cd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hpljx\" (UID: \"7215e114-f9ed-40cb-9de5-c48cef3995cd\") " pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:06:59.426816 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:59.426371    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7215e114-f9ed-40cb-9de5-c48cef3995cd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hpljx\" (UID: \"7215e114-f9ed-40cb-9de5-c48cef3995cd\") " pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:06:59.426816 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:59.426454    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 21:06:59.426816 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:59.426465    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 21:06:59.426816 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:59.426515    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7215e114-f9ed-40cb-9de5-c48cef3995cd-kube-rbac-proxy-cm podName:7215e114-f9ed-40cb-9de5-c48cef3995cd nodeName:}" failed. No retries permitted until 2026-07-20 21:07:03.42650091 +0000 UTC m=+264.330490467 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7215e114-f9ed-40cb-9de5-c48cef3995cd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hpljx" (UID: "7215e114-f9ed-40cb-9de5-c48cef3995cd") : configmap "kube-rbac-proxy" not found
Jul 20 21:06:59.426816 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:06:59.426531    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7215e114-f9ed-40cb-9de5-c48cef3995cd-insights-runtime-extractor-tls podName:7215e114-f9ed-40cb-9de5-c48cef3995cd nodeName:}" failed. No retries permitted until 2026-07-20 21:07:03.426523393 +0000 UTC m=+264.330512948 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7215e114-f9ed-40cb-9de5-c48cef3995cd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hpljx" (UID: "7215e114-f9ed-40cb-9de5-c48cef3995cd") : secret "insights-runtime-extractor-tls" not found
Jul 20 21:06:59.550279 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:59.550239    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-gvjfj" event={"ID":"79b8b7d5-2511-4da3-afe1-3a0c861f22ba","Type":"ContainerStarted","Data":"7fd6c93b0213a21e738194d28fc03f1108615b62b63f47d46fdfb85ad4bf52b5"}
Jul 20 21:06:59.550488 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:06:59.550292    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-gvjfj" event={"ID":"79b8b7d5-2511-4da3-afe1-3a0c861f22ba","Type":"ContainerStarted","Data":"ad72b7a00b162ef43503be1439813c099833e60701f6823b8c37e150d24d3418"}
Jul 20 21:07:00.035289 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:00.035250    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9g6f9_32060f49-8c0d-425a-a22f-8f032ddf9dea/serve-healthcheck-canary/0.log"
Jul 20 21:07:01.441166 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:01.441126    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-fz7g7"
Jul 20 21:07:01.441652 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:01.441196    2561 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-fz7g7"
Jul 20 21:07:01.441652 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:01.441634    2561 scope.go:117] "RemoveContainer" containerID="6271f4cd851fa35163db08bed72b801819ddbb1f43e30c9dcbb5ea2c45d81aa1"
Jul 20 21:07:01.441862 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:07:01.441838    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-fz7g7_openshift-console-operator(f3754f5e-54ec-4d60-9e1f-9290b4a82d07)\"" pod="openshift-console-operator/console-operator-565b7b569d-fz7g7" podUID="f3754f5e-54ec-4d60-9e1f-9290b4a82d07"
Jul 20 21:07:03.460444 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:03.460403    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7215e114-f9ed-40cb-9de5-c48cef3995cd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hpljx\" (UID: \"7215e114-f9ed-40cb-9de5-c48cef3995cd\") " pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:07:03.460866 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:03.460459    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7215e114-f9ed-40cb-9de5-c48cef3995cd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hpljx\" (UID: \"7215e114-f9ed-40cb-9de5-c48cef3995cd\") " pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:07:03.460866 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:07:03.460556    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 21:07:03.460866 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:07:03.460592    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 21:07:03.460866 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:07:03.460626    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7215e114-f9ed-40cb-9de5-c48cef3995cd-insights-runtime-extractor-tls podName:7215e114-f9ed-40cb-9de5-c48cef3995cd nodeName:}" failed. No retries permitted until 2026-07-20 21:07:11.460610613 +0000 UTC m=+272.364600171 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7215e114-f9ed-40cb-9de5-c48cef3995cd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hpljx" (UID: "7215e114-f9ed-40cb-9de5-c48cef3995cd") : secret "insights-runtime-extractor-tls" not found
Jul 20 21:07:03.460866 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:07:03.460641    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7215e114-f9ed-40cb-9de5-c48cef3995cd-kube-rbac-proxy-cm podName:7215e114-f9ed-40cb-9de5-c48cef3995cd nodeName:}" failed. No retries permitted until 2026-07-20 21:07:11.46063498 +0000 UTC m=+272.364624536 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7215e114-f9ed-40cb-9de5-c48cef3995cd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hpljx" (UID: "7215e114-f9ed-40cb-9de5-c48cef3995cd") : configmap "kube-rbac-proxy" not found
Jul 20 21:07:06.888806 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:06.888771    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3fb0dff-5b39-431b-959b-ba6636770fe4-metrics-certs\") pod \"router-default-64d7f6dfcb-zmq82\" (UID: \"e3fb0dff-5b39-431b-959b-ba6636770fe4\") " pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:07:06.888806 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:06.888809    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3fb0dff-5b39-431b-959b-ba6636770fe4-service-ca-bundle\") pod \"router-default-64d7f6dfcb-zmq82\" (UID: \"e3fb0dff-5b39-431b-959b-ba6636770fe4\") " pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:07:06.889538 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:06.889519    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3fb0dff-5b39-431b-959b-ba6636770fe4-service-ca-bundle\") pod \"router-default-64d7f6dfcb-zmq82\" (UID: \"e3fb0dff-5b39-431b-959b-ba6636770fe4\") " pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:07:06.891160 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:06.891141    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3fb0dff-5b39-431b-959b-ba6636770fe4-metrics-certs\") pod \"router-default-64d7f6dfcb-zmq82\" (UID: \"e3fb0dff-5b39-431b-959b-ba6636770fe4\") " pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:07:06.951463 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:06.951427    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:07:07.085489 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:07.085434    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-gvjfj" podStartSLOduration=9.085419304 podStartE2EDuration="9.085419304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:06:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:06:59.583509132 +0000 UTC m=+260.487498709" watchObservedRunningTime="2026-07-20 21:07:07.085419304 +0000 UTC m=+267.989408883"
Jul 20 21:07:07.086303 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:07.086281    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-64d7f6dfcb-zmq82"]
Jul 20 21:07:07.089094 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:07:07.089050    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3fb0dff_5b39_431b_959b_ba6636770fe4.slice/crio-44c37af9ec12ccce1c798df73bfa874e10edbc6915b83fb1a934be40d7a7f066 WatchSource:0}: Error finding container 44c37af9ec12ccce1c798df73bfa874e10edbc6915b83fb1a934be40d7a7f066: Status 404 returned error can't find the container with id 44c37af9ec12ccce1c798df73bfa874e10edbc6915b83fb1a934be40d7a7f066
Jul 20 21:07:07.580371 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:07.580333    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-64d7f6dfcb-zmq82" event={"ID":"e3fb0dff-5b39-431b-959b-ba6636770fe4","Type":"ContainerStarted","Data":"c09680d84ecb2da681d2a2135cc8e07b5a7457cc0e6db1e0546d9c3d73958642"}
Jul 20 21:07:07.580525 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:07.580391    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-64d7f6dfcb-zmq82" event={"ID":"e3fb0dff-5b39-431b-959b-ba6636770fe4","Type":"ContainerStarted","Data":"44c37af9ec12ccce1c798df73bfa874e10edbc6915b83fb1a934be40d7a7f066"}
Jul 20 21:07:07.606865 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:07.606805    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-64d7f6dfcb-zmq82" podStartSLOduration=17.60678653 podStartE2EDuration="17.60678653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:06:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:07:07.606524393 +0000 UTC m=+268.510513971" watchObservedRunningTime="2026-07-20 21:07:07.60678653 +0000 UTC m=+268.510776112"
Jul 20 21:07:07.952319 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:07.952231    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:07:07.954763 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:07.954739    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:07:08.584153 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:08.584116    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:07:08.585262 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:08.585239    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-64d7f6dfcb-zmq82"
Jul 20 21:07:11.526310 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:11.526277    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7215e114-f9ed-40cb-9de5-c48cef3995cd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hpljx\" (UID: \"7215e114-f9ed-40cb-9de5-c48cef3995cd\") " pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:07:11.526754 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:11.526322    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7215e114-f9ed-40cb-9de5-c48cef3995cd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hpljx\" (UID: \"7215e114-f9ed-40cb-9de5-c48cef3995cd\") " pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:07:11.526900 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:11.526878    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7215e114-f9ed-40cb-9de5-c48cef3995cd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hpljx\" (UID: \"7215e114-f9ed-40cb-9de5-c48cef3995cd\") " pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:07:11.528593 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:11.528575    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7215e114-f9ed-40cb-9de5-c48cef3995cd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hpljx\" (UID: \"7215e114-f9ed-40cb-9de5-c48cef3995cd\") " pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:07:11.616720 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:11.616673    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hpljx"
Jul 20 21:07:11.750499 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:11.750394    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hpljx"]
Jul 20 21:07:11.752983 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:07:11.752946    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7215e114_f9ed_40cb_9de5_c48cef3995cd.slice/crio-9965cc756185ef695c734786e41773d40bbd959d72b63839a591c6ac067c7668 WatchSource:0}: Error finding container 9965cc756185ef695c734786e41773d40bbd959d72b63839a591c6ac067c7668: Status 404 returned error can't find the container with id 9965cc756185ef695c734786e41773d40bbd959d72b63839a591c6ac067c7668
Jul 20 21:07:12.598304 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:12.598252    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hpljx" event={"ID":"7215e114-f9ed-40cb-9de5-c48cef3995cd","Type":"ContainerStarted","Data":"9965cc756185ef695c734786e41773d40bbd959d72b63839a591c6ac067c7668"}
Jul 20 21:07:12.720603 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:12.720575    2561 scope.go:117] "RemoveContainer" containerID="6271f4cd851fa35163db08bed72b801819ddbb1f43e30c9dcbb5ea2c45d81aa1"
Jul 20 21:07:13.604256 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:13.604229    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/1.log"
Jul 20 21:07:13.604679 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:13.604319    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-fz7g7" event={"ID":"f3754f5e-54ec-4d60-9e1f-9290b4a82d07","Type":"ContainerStarted","Data":"9c4afd6c00fba0dbe551891bb62729b5ee4bb609b7f6cf730f6fcfec15f7c613"}
Jul 20 21:07:13.604740 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:13.604718    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-fz7g7"
Jul 20 21:07:13.613209 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:13.613177    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-fz7g7"
Jul 20 21:07:13.667701 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:13.667645    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-fz7g7" podStartSLOduration=19.488262543 podStartE2EDuration="22.667629066s" totalImagesPullingTime="3.179366523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:06:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:07:13.667053461 +0000 UTC m=+274.571043040" watchObservedRunningTime="2026-07-20 21:07:13.667629066 +0000 UTC m=+274.571618685"
Jul 20 21:07:17.620187 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:17.620146    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hpljx" event={"ID":"7215e114-f9ed-40cb-9de5-c48cef3995cd","Type":"ContainerStarted","Data":"0872bc26919f8e071f0085845bfc87e1f0916283e243284fd93519212f52a8d6"}
Jul 20 21:07:20.065304 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.065268    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-ghswp"]
Jul 20 21:07:20.068903 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.068876    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-ghswp"
Jul 20 21:07:20.072648 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.072210    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 20 21:07:20.072880 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.072777    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 20 21:07:20.073270 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.073245    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-qbrg5\""
Jul 20 21:07:20.084173 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.084150    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-ghswp"]
Jul 20 21:07:20.140035 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.140004    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"]
Jul 20 21:07:20.143188 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.143165    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.147624 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.147606    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 20 21:07:20.147729 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.147650    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 20 21:07:20.147998 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.147973    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 20 21:07:20.148229 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.148213    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-24tzc\""
Jul 20 21:07:20.154476 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.154454    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 20 21:07:20.159231 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.159212    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"]
Jul 20 21:07:20.187601 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.187572    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c470afc9-11d3-43c5-b6fb-d599f7e34043-installation-pull-secrets\") pod \"image-registry-7b76f5dbfb-pc4k2\" (UID: \"c470afc9-11d3-43c5-b6fb-d599f7e34043\") " pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.187601 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.187603    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c470afc9-11d3-43c5-b6fb-d599f7e34043-bound-sa-token\") pod \"image-registry-7b76f5dbfb-pc4k2\" (UID: \"c470afc9-11d3-43c5-b6fb-d599f7e34043\") " pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.187759 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.187621    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c470afc9-11d3-43c5-b6fb-d599f7e34043-registry-certificates\") pod \"image-registry-7b76f5dbfb-pc4k2\" (UID: \"c470afc9-11d3-43c5-b6fb-d599f7e34043\") " pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.187759 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.187638    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c470afc9-11d3-43c5-b6fb-d599f7e34043-trusted-ca\") pod \"image-registry-7b76f5dbfb-pc4k2\" (UID: \"c470afc9-11d3-43c5-b6fb-d599f7e34043\") " pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.187759 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.187680    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c470afc9-11d3-43c5-b6fb-d599f7e34043-image-registry-private-configuration\") pod \"image-registry-7b76f5dbfb-pc4k2\" (UID: \"c470afc9-11d3-43c5-b6fb-d599f7e34043\") " pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.187759 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.187701    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c470afc9-11d3-43c5-b6fb-d599f7e34043-ca-trust-extracted\") pod \"image-registry-7b76f5dbfb-pc4k2\" (UID: \"c470afc9-11d3-43c5-b6fb-d599f7e34043\") " pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.187759 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.187742    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bsth\" (UniqueName: \"kubernetes.io/projected/c470afc9-11d3-43c5-b6fb-d599f7e34043-kube-api-access-2bsth\") pod \"image-registry-7b76f5dbfb-pc4k2\" (UID: \"c470afc9-11d3-43c5-b6fb-d599f7e34043\") " pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.187904 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.187763    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c470afc9-11d3-43c5-b6fb-d599f7e34043-registry-tls\") pod \"image-registry-7b76f5dbfb-pc4k2\" (UID: \"c470afc9-11d3-43c5-b6fb-d599f7e34043\") " pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.187904 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.187783    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57zqs\" (UniqueName: \"kubernetes.io/projected/ccb3a0dc-353e-4850-9ce2-ddd5bf113e38-kube-api-access-57zqs\") pod \"downloads-5bc67dcd46-ghswp\" (UID: \"ccb3a0dc-353e-4850-9ce2-ddd5bf113e38\") " pod="openshift-console/downloads-5bc67dcd46-ghswp"
Jul 20 21:07:20.288676 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.288642    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c470afc9-11d3-43c5-b6fb-d599f7e34043-installation-pull-secrets\") pod \"image-registry-7b76f5dbfb-pc4k2\" (UID: \"c470afc9-11d3-43c5-b6fb-d599f7e34043\") " pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.288676 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.288680    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c470afc9-11d3-43c5-b6fb-d599f7e34043-bound-sa-token\") pod \"image-registry-7b76f5dbfb-pc4k2\" (UID: \"c470afc9-11d3-43c5-b6fb-d599f7e34043\") " pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.288951 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.288707    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c470afc9-11d3-43c5-b6fb-d599f7e34043-registry-certificates\") pod \"image-registry-7b76f5dbfb-pc4k2\" (UID: \"c470afc9-11d3-43c5-b6fb-d599f7e34043\") " pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.288951 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.288734    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c470afc9-11d3-43c5-b6fb-d599f7e34043-trusted-ca\") pod \"image-registry-7b76f5dbfb-pc4k2\" (UID: \"c470afc9-11d3-43c5-b6fb-d599f7e34043\") " pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.288951 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.288760    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c470afc9-11d3-43c5-b6fb-d599f7e34043-image-registry-private-configuration\") pod \"image-registry-7b76f5dbfb-pc4k2\" (UID: \"c470afc9-11d3-43c5-b6fb-d599f7e34043\") " pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.288951 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.288787    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c470afc9-11d3-43c5-b6fb-d599f7e34043-ca-trust-extracted\") pod \"image-registry-7b76f5dbfb-pc4k2\" (UID: \"c470afc9-11d3-43c5-b6fb-d599f7e34043\") " pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.288951 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.288834    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bsth\" (UniqueName: \"kubernetes.io/projected/c470afc9-11d3-43c5-b6fb-d599f7e34043-kube-api-access-2bsth\") pod \"image-registry-7b76f5dbfb-pc4k2\" (UID: \"c470afc9-11d3-43c5-b6fb-d599f7e34043\") " pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.288951 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.288868    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c470afc9-11d3-43c5-b6fb-d599f7e34043-registry-tls\") pod \"image-registry-7b76f5dbfb-pc4k2\" (UID: \"c470afc9-11d3-43c5-b6fb-d599f7e34043\") " pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.288951 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.288899    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57zqs\" (UniqueName: \"kubernetes.io/projected/ccb3a0dc-353e-4850-9ce2-ddd5bf113e38-kube-api-access-57zqs\") pod \"downloads-5bc67dcd46-ghswp\" (UID: \"ccb3a0dc-353e-4850-9ce2-ddd5bf113e38\") " pod="openshift-console/downloads-5bc67dcd46-ghswp"
Jul 20 21:07:20.289442 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.289359    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c470afc9-11d3-43c5-b6fb-d599f7e34043-ca-trust-extracted\") pod \"image-registry-7b76f5dbfb-pc4k2\" (UID: \"c470afc9-11d3-43c5-b6fb-d599f7e34043\") " pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.289782 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.289753    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c470afc9-11d3-43c5-b6fb-d599f7e34043-registry-certificates\") pod \"image-registry-7b76f5dbfb-pc4k2\" (UID: \"c470afc9-11d3-43c5-b6fb-d599f7e34043\") " pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.289974 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.289957    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c470afc9-11d3-43c5-b6fb-d599f7e34043-trusted-ca\") pod \"image-registry-7b76f5dbfb-pc4k2\" (UID: \"c470afc9-11d3-43c5-b6fb-d599f7e34043\") " pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.291216 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.291198    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c470afc9-11d3-43c5-b6fb-d599f7e34043-installation-pull-secrets\") pod \"image-registry-7b76f5dbfb-pc4k2\" (UID: \"c470afc9-11d3-43c5-b6fb-d599f7e34043\") " pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.291600 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.291586    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c470afc9-11d3-43c5-b6fb-d599f7e34043-registry-tls\") pod \"image-registry-7b76f5dbfb-pc4k2\" (UID: \"c470afc9-11d3-43c5-b6fb-d599f7e34043\") " pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.291665 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.291634    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c470afc9-11d3-43c5-b6fb-d599f7e34043-image-registry-private-configuration\") pod \"image-registry-7b76f5dbfb-pc4k2\" (UID: \"c470afc9-11d3-43c5-b6fb-d599f7e34043\") " pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.298185 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.298152    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c470afc9-11d3-43c5-b6fb-d599f7e34043-bound-sa-token\") pod \"image-registry-7b76f5dbfb-pc4k2\" (UID: \"c470afc9-11d3-43c5-b6fb-d599f7e34043\") " pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.298652 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.298635    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57zqs\" (UniqueName: \"kubernetes.io/projected/ccb3a0dc-353e-4850-9ce2-ddd5bf113e38-kube-api-access-57zqs\") pod \"downloads-5bc67dcd46-ghswp\" (UID: \"ccb3a0dc-353e-4850-9ce2-ddd5bf113e38\") " pod="openshift-console/downloads-5bc67dcd46-ghswp"
Jul 20 21:07:20.298710 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.298695    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bsth\" (UniqueName: \"kubernetes.io/projected/c470afc9-11d3-43c5-b6fb-d599f7e34043-kube-api-access-2bsth\") pod \"image-registry-7b76f5dbfb-pc4k2\" (UID: \"c470afc9-11d3-43c5-b6fb-d599f7e34043\") " pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.382224 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.382138    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-ghswp"
Jul 20 21:07:20.454698 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.454658    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:20.514329 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.514228    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-ghswp"]
Jul 20 21:07:20.516831 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:07:20.516801    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccb3a0dc_353e_4850_9ce2_ddd5bf113e38.slice/crio-e8202da12113df790f7c1be5431c6eb04bca2b5da9a8f11de992120f0f6e3b37 WatchSource:0}: Error finding container e8202da12113df790f7c1be5431c6eb04bca2b5da9a8f11de992120f0f6e3b37: Status 404 returned error can't find the container with id e8202da12113df790f7c1be5431c6eb04bca2b5da9a8f11de992120f0f6e3b37
Jul 20 21:07:20.583514 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.583480    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"]
Jul 20 21:07:20.586025 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:07:20.586001    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc470afc9_11d3_43c5_b6fb_d599f7e34043.slice/crio-abca8df97b2a23caea8e3afa3620906d2827b52bc45a471726172a33681cc31a WatchSource:0}: Error finding container abca8df97b2a23caea8e3afa3620906d2827b52bc45a471726172a33681cc31a: Status 404 returned error can't find the container with id abca8df97b2a23caea8e3afa3620906d2827b52bc45a471726172a33681cc31a
Jul 20 21:07:20.631535 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.631510    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2" event={"ID":"c470afc9-11d3-43c5-b6fb-d599f7e34043","Type":"ContainerStarted","Data":"abca8df97b2a23caea8e3afa3620906d2827b52bc45a471726172a33681cc31a"}
Jul 20 21:07:20.633133 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.633067    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hpljx" event={"ID":"7215e114-f9ed-40cb-9de5-c48cef3995cd","Type":"ContainerStarted","Data":"51a265bb13def7bbea319b55f0dd6d52467297bf125701447ad28d0147a38b21"}
Jul 20 21:07:20.634189 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:20.634168    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-ghswp" event={"ID":"ccb3a0dc-353e-4850-9ce2-ddd5bf113e38","Type":"ContainerStarted","Data":"e8202da12113df790f7c1be5431c6eb04bca2b5da9a8f11de992120f0f6e3b37"}
Jul 20 21:07:21.296591 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:07:21.296544    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-prms6" podUID="8e2b83ca-b2c3-4899-92ed-16890097cd04"
Jul 20 21:07:21.641096 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:21.640406    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-prms6"
Jul 20 21:07:21.641096 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:21.640772    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2" event={"ID":"c470afc9-11d3-43c5-b6fb-d599f7e34043","Type":"ContainerStarted","Data":"77ce21245dabcce4428aa21b9aaf7981d5f28c671c6def52de4f864270d3fa08"}
Jul 20 21:07:21.641096 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:21.640848    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:21.665571 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:21.665513    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2" podStartSLOduration=1.665489946 podStartE2EDuration="1.665489946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:07:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:07:21.663259239 +0000 UTC m=+282.567248837" watchObservedRunningTime="2026-07-20 21:07:21.665489946 +0000 UTC m=+282.569479525"
Jul 20 21:07:24.002114 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.001972    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-6t8bd"]
Jul 20 21:07:24.022520 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.022457    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-6t8bd"]
Jul 20 21:07:24.022690 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.022581    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-6t8bd"
Jul 20 21:07:24.025600 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.025573    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 20 21:07:24.025732 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.025614    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 20 21:07:24.026532 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.026506    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 20 21:07:24.026532 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.026522    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 20 21:07:24.026997 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.026976    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-zvgrv\""
Jul 20 21:07:24.027244 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.027226    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 20 21:07:24.114815 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.114585    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c71af73b-6f46-46a3-b6ca-f1ddb5d4f240-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-6t8bd\" (UID: \"c71af73b-6f46-46a3-b6ca-f1ddb5d4f240\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-6t8bd"
Jul 20 21:07:24.114815 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.114635    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c71af73b-6f46-46a3-b6ca-f1ddb5d4f240-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-6t8bd\" (UID: \"c71af73b-6f46-46a3-b6ca-f1ddb5d4f240\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-6t8bd"
Jul 20 21:07:24.114815 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.114682    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ljh4\" (UniqueName: \"kubernetes.io/projected/c71af73b-6f46-46a3-b6ca-f1ddb5d4f240-kube-api-access-5ljh4\") pod \"prometheus-operator-5945b5cfd-6t8bd\" (UID: \"c71af73b-6f46-46a3-b6ca-f1ddb5d4f240\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-6t8bd"
Jul 20 21:07:24.114815 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.114762    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c71af73b-6f46-46a3-b6ca-f1ddb5d4f240-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-6t8bd\" (UID: \"c71af73b-6f46-46a3-b6ca-f1ddb5d4f240\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-6t8bd"
Jul 20 21:07:24.215968 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.215935    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c71af73b-6f46-46a3-b6ca-f1ddb5d4f240-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-6t8bd\" (UID: \"c71af73b-6f46-46a3-b6ca-f1ddb5d4f240\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-6t8bd"
Jul 20 21:07:24.215968 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.215973    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c71af73b-6f46-46a3-b6ca-f1ddb5d4f240-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-6t8bd\" (UID: \"c71af73b-6f46-46a3-b6ca-f1ddb5d4f240\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-6t8bd"
Jul 20 21:07:24.216221 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.216011    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ljh4\" (UniqueName: \"kubernetes.io/projected/c71af73b-6f46-46a3-b6ca-f1ddb5d4f240-kube-api-access-5ljh4\") pod \"prometheus-operator-5945b5cfd-6t8bd\" (UID: \"c71af73b-6f46-46a3-b6ca-f1ddb5d4f240\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-6t8bd"
Jul 20 21:07:24.216221 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.216057    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c71af73b-6f46-46a3-b6ca-f1ddb5d4f240-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-6t8bd\" (UID: \"c71af73b-6f46-46a3-b6ca-f1ddb5d4f240\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-6t8bd"
Jul 20 21:07:24.216801 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.216773    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c71af73b-6f46-46a3-b6ca-f1ddb5d4f240-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-6t8bd\" (UID: \"c71af73b-6f46-46a3-b6ca-f1ddb5d4f240\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-6t8bd"
Jul 20 21:07:24.218608 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.218582    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c71af73b-6f46-46a3-b6ca-f1ddb5d4f240-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-6t8bd\" (UID: \"c71af73b-6f46-46a3-b6ca-f1ddb5d4f240\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-6t8bd"
Jul 20 21:07:24.218968 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.218943    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c71af73b-6f46-46a3-b6ca-f1ddb5d4f240-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-6t8bd\" (UID: \"c71af73b-6f46-46a3-b6ca-f1ddb5d4f240\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-6t8bd"
Jul 20 21:07:24.224826 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.224781    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ljh4\" (UniqueName: \"kubernetes.io/projected/c71af73b-6f46-46a3-b6ca-f1ddb5d4f240-kube-api-access-5ljh4\") pod \"prometheus-operator-5945b5cfd-6t8bd\" (UID: \"c71af73b-6f46-46a3-b6ca-f1ddb5d4f240\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-6t8bd"
Jul 20 21:07:24.335117 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.335086    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-6t8bd"
Jul 20 21:07:24.417483 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.417442    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls\") pod \"dns-default-prms6\" (UID: \"8e2b83ca-b2c3-4899-92ed-16890097cd04\") " pod="openshift-dns/dns-default-prms6"
Jul 20 21:07:24.420304 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.420276    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e2b83ca-b2c3-4899-92ed-16890097cd04-metrics-tls\") pod \"dns-default-prms6\" (UID: \"8e2b83ca-b2c3-4899-92ed-16890097cd04\") " pod="openshift-dns/dns-default-prms6"
Jul 20 21:07:24.644325 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.644301    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-n29b9\""
Jul 20 21:07:24.651916 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.651865    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-prms6"
Jul 20 21:07:24.713855 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.713702    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-6t8bd"]
Jul 20 21:07:24.720441 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:07:24.720411    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc71af73b_6f46_46a3_b6ca_f1ddb5d4f240.slice/crio-7e16db0872740a47c81b97a5b61408feca8a46bc4fde1ba057788d1e1414d5fe WatchSource:0}: Error finding container 7e16db0872740a47c81b97a5b61408feca8a46bc4fde1ba057788d1e1414d5fe: Status 404 returned error can't find the container with id 7e16db0872740a47c81b97a5b61408feca8a46bc4fde1ba057788d1e1414d5fe
Jul 20 21:07:24.801739 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:24.801708    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-prms6"]
Jul 20 21:07:24.805559 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:07:24.805513    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e2b83ca_b2c3_4899_92ed_16890097cd04.slice/crio-a4c7fabd8d0e9a5307b1583c96e2dfd6dec708e2d8a432b0b18bbe0cd81ea1fa WatchSource:0}: Error finding container a4c7fabd8d0e9a5307b1583c96e2dfd6dec708e2d8a432b0b18bbe0cd81ea1fa: Status 404 returned error can't find the container with id a4c7fabd8d0e9a5307b1583c96e2dfd6dec708e2d8a432b0b18bbe0cd81ea1fa
Jul 20 21:07:25.659620 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:25.659575    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hpljx" event={"ID":"7215e114-f9ed-40cb-9de5-c48cef3995cd","Type":"ContainerStarted","Data":"a21c6d63042c1c1c65556777b24a77048115bb34fccbfc11499821adf8f9963c"}
Jul 20 21:07:25.661222 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:25.661191    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-prms6" event={"ID":"8e2b83ca-b2c3-4899-92ed-16890097cd04","Type":"ContainerStarted","Data":"a4c7fabd8d0e9a5307b1583c96e2dfd6dec708e2d8a432b0b18bbe0cd81ea1fa"}
Jul 20 21:07:25.662481 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:25.662457    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-6t8bd" event={"ID":"c71af73b-6f46-46a3-b6ca-f1ddb5d4f240","Type":"ContainerStarted","Data":"7e16db0872740a47c81b97a5b61408feca8a46bc4fde1ba057788d1e1414d5fe"}
Jul 20 21:07:25.678285 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:25.678230    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hpljx" podStartSLOduration=17.941288308 podStartE2EDuration="30.678208557s" totalImagesPullingTime="12.736920249s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 21:06:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:07:25.677485837 +0000 UTC m=+286.581475470" watchObservedRunningTime="2026-07-20 21:07:25.678208557 +0000 UTC m=+286.582198127"
Jul 20 21:07:27.671546 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:27.671505    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-prms6" event={"ID":"8e2b83ca-b2c3-4899-92ed-16890097cd04","Type":"ContainerStarted","Data":"4584abd576b0cb224824c324ed714f51f7b1a32910446cc76f9985c40251c0d3"}
Jul 20 21:07:27.672000 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:27.671598    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-prms6" event={"ID":"8e2b83ca-b2c3-4899-92ed-16890097cd04","Type":"ContainerStarted","Data":"66e3db1d4d5b44e6f6cec79307636a16fe054f53694a7e96b8b438b0876fb376"}
Jul 20 21:07:27.672000 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:27.671627    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-prms6"
Jul 20 21:07:27.673130 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:27.673110    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-6t8bd" event={"ID":"c71af73b-6f46-46a3-b6ca-f1ddb5d4f240","Type":"ContainerStarted","Data":"7f38ffba8d0232e3d806bd1ccdf5dc7388bb435ad02b363a329f331082548ef8"}
Jul 20 21:07:27.673172 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:27.673151    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-6t8bd" event={"ID":"c71af73b-6f46-46a3-b6ca-f1ddb5d4f240","Type":"ContainerStarted","Data":"b4b609cc0d8d5dfd907c8f0a7fddf3224d72bb9a9b21db7dee20edc9e7e0f8f5"}
Jul 20 21:07:27.689204 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:27.689154    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-prms6" podStartSLOduration=251.62895574 podStartE2EDuration="4m13.689137997s" totalImagesPullingTime="2.060182257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:03:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:07:27.688766537 +0000 UTC m=+288.592756114" watchObservedRunningTime="2026-07-20 21:07:27.689137997 +0000 UTC m=+288.593127575"
Jul 20 21:07:27.707568 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:27.707524    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-6t8bd" podStartSLOduration=2.609233309 podStartE2EDuration="4.707507829s" totalImagesPullingTime="2.09827452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:07:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:07:27.706138787 +0000 UTC m=+288.610128388" watchObservedRunningTime="2026-07-20 21:07:27.707507829 +0000 UTC m=+288.611497404"
Jul 20 21:07:29.360693 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.360649    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-kdnkm"]
Jul 20 21:07:29.376914 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.376792    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kdnkm"
Jul 20 21:07:29.379456 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.379386    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-kdnkm"]
Jul 20 21:07:29.381946 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.381898    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 20 21:07:29.382066 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.381985    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-6hnv6\""
Jul 20 21:07:29.382196 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.382178    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 20 21:07:29.391441 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.391410    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-4csfn"]
Jul 20 21:07:29.393885 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.393865    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.396844 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.396708    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8pvq7\""
Jul 20 21:07:29.397307 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.396980    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 20 21:07:29.397307 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.397066    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 20 21:07:29.397307 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.397250    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 20 21:07:29.464945 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.464406    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/83bc0281-5b82-42cb-92a6-3652a24925cc-node-exporter-tls\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.464945 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.464455    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/22e52d64-9dd4-4aa4-9b40-c3c3b2230975-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-kdnkm\" (UID: \"22e52d64-9dd4-4aa4-9b40-c3c3b2230975\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kdnkm"
Jul 20 21:07:29.464945 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.464484    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/83bc0281-5b82-42cb-92a6-3652a24925cc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.464945 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.464509    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8kqp\" (UniqueName: \"kubernetes.io/projected/83bc0281-5b82-42cb-92a6-3652a24925cc-kube-api-access-l8kqp\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.464945 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.464536    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83bc0281-5b82-42cb-92a6-3652a24925cc-metrics-client-ca\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.464945 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.464560    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/83bc0281-5b82-42cb-92a6-3652a24925cc-sys\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.464945 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.464588    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/83bc0281-5b82-42cb-92a6-3652a24925cc-node-exporter-accelerators-collector-config\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.464945 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.464612    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22e52d64-9dd4-4aa4-9b40-c3c3b2230975-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-kdnkm\" (UID: \"22e52d64-9dd4-4aa4-9b40-c3c3b2230975\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kdnkm"
Jul 20 21:07:29.464945 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.464658    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/83bc0281-5b82-42cb-92a6-3652a24925cc-root\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.464945 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.464679    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/83bc0281-5b82-42cb-92a6-3652a24925cc-node-exporter-wtmp\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.464945 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.464710    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ng6n\" (UniqueName: \"kubernetes.io/projected/22e52d64-9dd4-4aa4-9b40-c3c3b2230975-kube-api-access-6ng6n\") pod \"openshift-state-metrics-79f6958cdf-kdnkm\" (UID: \"22e52d64-9dd4-4aa4-9b40-c3c3b2230975\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kdnkm"
Jul 20 21:07:29.464945 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.464736    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22e52d64-9dd4-4aa4-9b40-c3c3b2230975-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-kdnkm\" (UID: \"22e52d64-9dd4-4aa4-9b40-c3c3b2230975\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kdnkm"
Jul 20 21:07:29.464945 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.464777    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/83bc0281-5b82-42cb-92a6-3652a24925cc-node-exporter-textfile\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.566209 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.566173    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22e52d64-9dd4-4aa4-9b40-c3c3b2230975-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-kdnkm\" (UID: \"22e52d64-9dd4-4aa4-9b40-c3c3b2230975\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kdnkm"
Jul 20 21:07:29.566394 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.566224    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/83bc0281-5b82-42cb-92a6-3652a24925cc-node-exporter-textfile\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.566394 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.566285    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/83bc0281-5b82-42cb-92a6-3652a24925cc-node-exporter-tls\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.566394 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.566312    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/22e52d64-9dd4-4aa4-9b40-c3c3b2230975-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-kdnkm\" (UID: \"22e52d64-9dd4-4aa4-9b40-c3c3b2230975\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kdnkm"
Jul 20 21:07:29.566394 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.566339    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/83bc0281-5b82-42cb-92a6-3652a24925cc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.566394 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.566363    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8kqp\" (UniqueName: \"kubernetes.io/projected/83bc0281-5b82-42cb-92a6-3652a24925cc-kube-api-access-l8kqp\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.566394 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.566389    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83bc0281-5b82-42cb-92a6-3652a24925cc-metrics-client-ca\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.566714 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.566412    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/83bc0281-5b82-42cb-92a6-3652a24925cc-sys\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.566714 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.566438    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/83bc0281-5b82-42cb-92a6-3652a24925cc-node-exporter-accelerators-collector-config\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.566714 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.566464    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22e52d64-9dd4-4aa4-9b40-c3c3b2230975-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-kdnkm\" (UID: \"22e52d64-9dd4-4aa4-9b40-c3c3b2230975\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kdnkm"
Jul 20 21:07:29.566714 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.566514    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/83bc0281-5b82-42cb-92a6-3652a24925cc-root\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.566714 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.566537    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/83bc0281-5b82-42cb-92a6-3652a24925cc-node-exporter-wtmp\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.566714 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.566568    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ng6n\" (UniqueName: \"kubernetes.io/projected/22e52d64-9dd4-4aa4-9b40-c3c3b2230975-kube-api-access-6ng6n\") pod \"openshift-state-metrics-79f6958cdf-kdnkm\" (UID: \"22e52d64-9dd4-4aa4-9b40-c3c3b2230975\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kdnkm"
Jul 20 21:07:29.567018 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.566816    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/83bc0281-5b82-42cb-92a6-3652a24925cc-node-exporter-textfile\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.567018 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.566893    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/83bc0281-5b82-42cb-92a6-3652a24925cc-sys\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.567992 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.567233    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/83bc0281-5b82-42cb-92a6-3652a24925cc-root\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.567992 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.567398    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/83bc0281-5b82-42cb-92a6-3652a24925cc-node-exporter-wtmp\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.567992 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.567423    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/83bc0281-5b82-42cb-92a6-3652a24925cc-node-exporter-accelerators-collector-config\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.567992 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.567939    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22e52d64-9dd4-4aa4-9b40-c3c3b2230975-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-kdnkm\" (UID: \"22e52d64-9dd4-4aa4-9b40-c3c3b2230975\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kdnkm"
Jul 20 21:07:29.567992 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.567951    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83bc0281-5b82-42cb-92a6-3652a24925cc-metrics-client-ca\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.570152 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.570127    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/22e52d64-9dd4-4aa4-9b40-c3c3b2230975-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-kdnkm\" (UID: \"22e52d64-9dd4-4aa4-9b40-c3c3b2230975\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kdnkm"
Jul 20 21:07:29.570552 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.570520    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/83bc0281-5b82-42cb-92a6-3652a24925cc-node-exporter-tls\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.570552 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.570543    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22e52d64-9dd4-4aa4-9b40-c3c3b2230975-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-kdnkm\" (UID: \"22e52d64-9dd4-4aa4-9b40-c3c3b2230975\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kdnkm"
Jul 20 21:07:29.570723 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.570672    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/83bc0281-5b82-42cb-92a6-3652a24925cc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.577138 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.577093    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ng6n\" (UniqueName: \"kubernetes.io/projected/22e52d64-9dd4-4aa4-9b40-c3c3b2230975-kube-api-access-6ng6n\") pod \"openshift-state-metrics-79f6958cdf-kdnkm\" (UID: \"22e52d64-9dd4-4aa4-9b40-c3c3b2230975\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kdnkm"
Jul 20 21:07:29.577846 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.577826    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8kqp\" (UniqueName: \"kubernetes.io/projected/83bc0281-5b82-42cb-92a6-3652a24925cc-kube-api-access-l8kqp\") pod \"node-exporter-4csfn\" (UID: \"83bc0281-5b82-42cb-92a6-3652a24925cc\") " pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.690856 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.690695    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kdnkm"
Jul 20 21:07:29.705781 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.705754    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4csfn"
Jul 20 21:07:29.860492 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:29.860458    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-kdnkm"]
Jul 20 21:07:29.864156 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:07:29.864128    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22e52d64_9dd4_4aa4_9b40_c3c3b2230975.slice/crio-d3dd6b2fdda3437f1ef9bd2eb27f379a541c45542c56a60f11fe5fdbd31cf7d0 WatchSource:0}: Error finding container d3dd6b2fdda3437f1ef9bd2eb27f379a541c45542c56a60f11fe5fdbd31cf7d0: Status 404 returned error can't find the container with id d3dd6b2fdda3437f1ef9bd2eb27f379a541c45542c56a60f11fe5fdbd31cf7d0
Jul 20 21:07:30.425338 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.425275    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 20 21:07:30.428801 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.428651    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.431223 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.431182    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 20 21:07:30.431505 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.431487    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 20 21:07:30.431660 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.431620    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 20 21:07:30.431786 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.431769    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 20 21:07:30.431941 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.431918    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 20 21:07:30.432282 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.432202    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 20 21:07:30.432594 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.432260    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 20 21:07:30.432594 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.432464    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 20 21:07:30.432728 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.432655    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-j5llk\""
Jul 20 21:07:30.432846 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.432830    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 20 21:07:30.448724 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.448692    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 20 21:07:30.473732 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.473689    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/40dcb9c0-2c68-4d9e-8f5f-411863c640da-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.473910 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.473750    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-config-volume\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.473910 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.473789    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-web-config\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.473910 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.473844    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.473910 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.473882    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40dcb9c0-2c68-4d9e-8f5f-411863c640da-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.474210 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.473916    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/40dcb9c0-2c68-4d9e-8f5f-411863c640da-tls-assets\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.474210 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.473948    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.474210 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.474002    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.474210 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.474046    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/40dcb9c0-2c68-4d9e-8f5f-411863c640da-config-out\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.474210 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.474066    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.474210 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.474118    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40dcb9c0-2c68-4d9e-8f5f-411863c640da-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.474210 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.474156    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p8cp\" (UniqueName: \"kubernetes.io/projected/40dcb9c0-2c68-4d9e-8f5f-411863c640da-kube-api-access-5p8cp\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.474210 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.474185    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.576026 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.575986    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/40dcb9c0-2c68-4d9e-8f5f-411863c640da-tls-assets\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.576227 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.576041    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.576227 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.576096    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.576227 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.576160    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/40dcb9c0-2c68-4d9e-8f5f-411863c640da-config-out\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.576227 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.576192    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.576227 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.576222    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40dcb9c0-2c68-4d9e-8f5f-411863c640da-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.576489 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.576257    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p8cp\" (UniqueName: \"kubernetes.io/projected/40dcb9c0-2c68-4d9e-8f5f-411863c640da-kube-api-access-5p8cp\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.576489 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.576284    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.576489 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.576333    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/40dcb9c0-2c68-4d9e-8f5f-411863c640da-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.576489 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.576374    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-config-volume\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.576489 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.576405    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-web-config\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.576489 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.576437    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.576489 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.576483    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40dcb9c0-2c68-4d9e-8f5f-411863c640da-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.580103 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.580013    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/40dcb9c0-2c68-4d9e-8f5f-411863c640da-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.580849 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:07:30.580307    2561 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 20 21:07:30.580849 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:07:30.580373    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-main-tls podName:40dcb9c0-2c68-4d9e-8f5f-411863c640da nodeName:}" failed. No retries permitted until 2026-07-20 21:07:31.080353532 +0000 UTC m=+291.984343093 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "40dcb9c0-2c68-4d9e-8f5f-411863c640da") : secret "alertmanager-main-tls" not found
Jul 20 21:07:30.582098 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.582023    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40dcb9c0-2c68-4d9e-8f5f-411863c640da-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.582198 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.582159    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40dcb9c0-2c68-4d9e-8f5f-411863c640da-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.583365 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.583322    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/40dcb9c0-2c68-4d9e-8f5f-411863c640da-config-out\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.586286 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.586245    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p8cp\" (UniqueName: \"kubernetes.io/projected/40dcb9c0-2c68-4d9e-8f5f-411863c640da-kube-api-access-5p8cp\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.587940 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.587894    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.587940 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.587913    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-config-volume\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.588126 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.588000    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.588356 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.588331    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.588600 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.588583    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/40dcb9c0-2c68-4d9e-8f5f-411863c640da-tls-assets\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.589454 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.589433    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.589683 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.589645    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-web-config\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:30.690704 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.690612    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4csfn" event={"ID":"83bc0281-5b82-42cb-92a6-3652a24925cc","Type":"ContainerStarted","Data":"757d0c34f333e8d6c100ba5369a286cdec9da5c0c5ce9d4089b6ac9b763ef716"}
Jul 20 21:07:30.693340 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.693310    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kdnkm" event={"ID":"22e52d64-9dd4-4aa4-9b40-c3c3b2230975","Type":"ContainerStarted","Data":"b7025b083a6452b548f2c18413be874cd33027fb3b94a95ce0f814c8a4a41f01"}
Jul 20 21:07:30.693479 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.693374    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kdnkm" event={"ID":"22e52d64-9dd4-4aa4-9b40-c3c3b2230975","Type":"ContainerStarted","Data":"cf980597654effa47e5e52f7a33a97f8faac07ab2612c5d222908582f1d999ed"}
Jul 20 21:07:30.693479 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:30.693406    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kdnkm" event={"ID":"22e52d64-9dd4-4aa4-9b40-c3c3b2230975","Type":"ContainerStarted","Data":"d3dd6b2fdda3437f1ef9bd2eb27f379a541c45542c56a60f11fe5fdbd31cf7d0"}
Jul 20 21:07:31.081951 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:31.081912    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:31.085054 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:31.085018    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:31.343870 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:31.343833    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:07:31.491206 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:31.491169    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 20 21:07:31.494512 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:07:31.494481    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40dcb9c0_2c68_4d9e_8f5f_411863c640da.slice/crio-64f9add49610064f1153bd7d5b6f60b9c68de12ee0e36dffc6c76e4da5d99b2b WatchSource:0}: Error finding container 64f9add49610064f1153bd7d5b6f60b9c68de12ee0e36dffc6c76e4da5d99b2b: Status 404 returned error can't find the container with id 64f9add49610064f1153bd7d5b6f60b9c68de12ee0e36dffc6c76e4da5d99b2b
Jul 20 21:07:31.699492 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:31.699451    2561 generic.go:356] "Generic (PLEG): container finished" podID="83bc0281-5b82-42cb-92a6-3652a24925cc" containerID="6012589553e508291b33e7fff61feb4c5c61690b8d77d9db77030428f44c9869" exitCode=0
Jul 20 21:07:31.699657 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:31.699544    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4csfn" event={"ID":"83bc0281-5b82-42cb-92a6-3652a24925cc","Type":"ContainerDied","Data":"6012589553e508291b33e7fff61feb4c5c61690b8d77d9db77030428f44c9869"}
Jul 20 21:07:31.701150 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:31.701118    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40dcb9c0-2c68-4d9e-8f5f-411863c640da","Type":"ContainerStarted","Data":"64f9add49610064f1153bd7d5b6f60b9c68de12ee0e36dffc6c76e4da5d99b2b"}
Jul 20 21:07:31.703662 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:31.703476    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kdnkm" event={"ID":"22e52d64-9dd4-4aa4-9b40-c3c3b2230975","Type":"ContainerStarted","Data":"0d69cc67b733c3a32c91723baaa85cfa000bae2694618856e818ced7954d2689"}
Jul 20 21:07:31.739962 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:31.739914    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-kdnkm" podStartSLOduration=1.512858507 podStartE2EDuration="2.739894309s" totalImagesPullingTime="1.227035802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:07:31.738511597 +0000 UTC m=+292.642501189" watchObservedRunningTime="2026-07-20 21:07:31.739894309 +0000 UTC m=+292.643883888"
Jul 20 21:07:32.712246 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:32.712140    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4csfn" event={"ID":"83bc0281-5b82-42cb-92a6-3652a24925cc","Type":"ContainerStarted","Data":"a3e7b8c519230eebc440b6e85f95c3d87009eb591e543ca4690f0ddc3f4de3ad"}
Jul 20 21:07:32.712246 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:32.712216    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4csfn" event={"ID":"83bc0281-5b82-42cb-92a6-3652a24925cc","Type":"ContainerStarted","Data":"6b3600bf94777488f39bd05efe254d0b95eaa81a5b36452ddcd6cd9d3bd4fb23"}
Jul 20 21:07:32.737738 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:32.737668    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-4csfn" podStartSLOduration=2.513260063 podStartE2EDuration="3.73764899s" totalImagesPullingTime="1.14466325s" totalInitContainerRuntime="79.725677ms" isStatefulPod=true podCreationTimestamp="2026-07-20 21:07:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:07:32.736254457 +0000 UTC m=+293.640244026" watchObservedRunningTime="2026-07-20 21:07:32.73764899 +0000 UTC m=+293.641638548"
Jul 20 21:07:33.717828 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:33.717789    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40dcb9c0-2c68-4d9e-8f5f-411863c640da","Type":"ContainerStarted","Data":"1bd226643cae56a96bd43b76d48155de9b1598642f9916e28ab913f4362e02a2"}
Jul 20 21:07:34.721635 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:07:34.721589    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-khqfb" podUID="3f6a3926-e94e-4981-9952-3ec7307a1e48"
Jul 20 21:07:37.680677 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:37.680641    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-prms6"
Jul 20 21:07:40.302609 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:40.302584    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/1.log"
Jul 20 21:07:40.302609 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:40.302601    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/1.log"
Jul 20 21:07:40.312543 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:40.312521    2561 kubelet.go:1648] "Image garbage collection succeeded"
Jul 20 21:07:40.750823 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:40.750786    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-ghswp" event={"ID":"ccb3a0dc-353e-4850-9ce2-ddd5bf113e38","Type":"ContainerStarted","Data":"d6494c758334d1f76d88b6b806fa48e64449690a898d73f99c649c4bd6641299"}
Jul 20 21:07:40.751003 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:40.750852    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-ghswp"
Jul 20 21:07:40.752469 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:40.752438    2561 generic.go:356] "Generic (PLEG): container finished" podID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerID="1bd226643cae56a96bd43b76d48155de9b1598642f9916e28ab913f4362e02a2" exitCode=0
Jul 20 21:07:40.752600 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:40.752518    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40dcb9c0-2c68-4d9e-8f5f-411863c640da","Type":"ContainerDied","Data":"1bd226643cae56a96bd43b76d48155de9b1598642f9916e28ab913f4362e02a2"}
Jul 20 21:07:40.752687 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:40.752662    2561 patch_prober.go:28] interesting pod/downloads-5bc67dcd46-ghswp container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.15:8080/\": dial tcp 10.133.0.15:8080: connect: connection refused" start-of-body=
Jul 20 21:07:40.752748 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:40.752721    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-5bc67dcd46-ghswp" podUID="ccb3a0dc-353e-4850-9ce2-ddd5bf113e38" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.15:8080/\": dial tcp 10.133.0.15:8080: connect: connection refused"
Jul 20 21:07:40.753686 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:40.753672    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 21:07:40.771590 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:40.771543    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-ghswp" podStartSLOduration=0.67987423 podStartE2EDuration="20.77150449s" totalImagesPullingTime="20.09163026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:07:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:07:40.77127109 +0000 UTC m=+301.675260678" watchObservedRunningTime="2026-07-20 21:07:40.77150449 +0000 UTC m=+301.675494046"
Jul 20 21:07:41.769237 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:41.769196    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-ghswp"
Jul 20 21:07:41.995510 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:41.995405    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8549799cff-5qgxt"]
Jul 20 21:07:42.059799 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.058494    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8549799cff-5qgxt"]
Jul 20 21:07:42.059799 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.058529    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:42.061818 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.061715    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-98t82\""
Jul 20 21:07:42.062299 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.062276    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 20 21:07:42.062415 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.062336    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 20 21:07:42.062563 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.062536    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 20 21:07:42.062647 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.062623    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 20 21:07:42.062758 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.062536    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 20 21:07:42.067654 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.067630    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 20 21:07:42.191358 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.191315    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9acc6489-80b6-4a49-b7cc-ff551323d1ec-service-ca\") pod \"console-8549799cff-5qgxt\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") " pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:42.191550 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.191370    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9acc6489-80b6-4a49-b7cc-ff551323d1ec-oauth-serving-cert\") pod \"console-8549799cff-5qgxt\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") " pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:42.191550 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.191461    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9acc6489-80b6-4a49-b7cc-ff551323d1ec-console-config\") pod \"console-8549799cff-5qgxt\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") " pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:42.191550 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.191510    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9acc6489-80b6-4a49-b7cc-ff551323d1ec-console-oauth-config\") pod \"console-8549799cff-5qgxt\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") " pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:42.191689 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.191619    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hlfm\" (UniqueName: \"kubernetes.io/projected/9acc6489-80b6-4a49-b7cc-ff551323d1ec-kube-api-access-4hlfm\") pod \"console-8549799cff-5qgxt\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") " pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:42.191739 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.191698    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9acc6489-80b6-4a49-b7cc-ff551323d1ec-trusted-ca-bundle\") pod \"console-8549799cff-5qgxt\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") " pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:42.191787 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.191737    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9acc6489-80b6-4a49-b7cc-ff551323d1ec-console-serving-cert\") pod \"console-8549799cff-5qgxt\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") " pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:42.292336 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.292258    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9acc6489-80b6-4a49-b7cc-ff551323d1ec-console-config\") pod \"console-8549799cff-5qgxt\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") " pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:42.292336 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.292324    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9acc6489-80b6-4a49-b7cc-ff551323d1ec-console-oauth-config\") pod \"console-8549799cff-5qgxt\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") " pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:42.292336 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.292378    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hlfm\" (UniqueName: \"kubernetes.io/projected/9acc6489-80b6-4a49-b7cc-ff551323d1ec-kube-api-access-4hlfm\") pod \"console-8549799cff-5qgxt\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") " pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:42.292696 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.292427    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9acc6489-80b6-4a49-b7cc-ff551323d1ec-trusted-ca-bundle\") pod \"console-8549799cff-5qgxt\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") " pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:42.292696 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.292457    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9acc6489-80b6-4a49-b7cc-ff551323d1ec-console-serving-cert\") pod \"console-8549799cff-5qgxt\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") " pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:42.292696 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.292501    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9acc6489-80b6-4a49-b7cc-ff551323d1ec-service-ca\") pod \"console-8549799cff-5qgxt\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") " pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:42.292696 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.292531    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9acc6489-80b6-4a49-b7cc-ff551323d1ec-oauth-serving-cert\") pod \"console-8549799cff-5qgxt\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") " pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:42.294563 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.293910    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9acc6489-80b6-4a49-b7cc-ff551323d1ec-oauth-serving-cert\") pod \"console-8549799cff-5qgxt\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") " pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:42.294563 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.294020    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9acc6489-80b6-4a49-b7cc-ff551323d1ec-console-config\") pod \"console-8549799cff-5qgxt\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") " pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:42.294563 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.294377    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9acc6489-80b6-4a49-b7cc-ff551323d1ec-service-ca\") pod \"console-8549799cff-5qgxt\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") " pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:42.295369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.295326    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9acc6489-80b6-4a49-b7cc-ff551323d1ec-trusted-ca-bundle\") pod \"console-8549799cff-5qgxt\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") " pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:42.296322 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.296301    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9acc6489-80b6-4a49-b7cc-ff551323d1ec-console-serving-cert\") pod \"console-8549799cff-5qgxt\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") " pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:42.296788 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.296765    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9acc6489-80b6-4a49-b7cc-ff551323d1ec-console-oauth-config\") pod \"console-8549799cff-5qgxt\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") " pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:42.302449 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.302421    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hlfm\" (UniqueName: \"kubernetes.io/projected/9acc6489-80b6-4a49-b7cc-ff551323d1ec-kube-api-access-4hlfm\") pod \"console-8549799cff-5qgxt\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") " pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:42.372631 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.372545    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:42.650042 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.649955    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7b76f5dbfb-pc4k2"
Jul 20 21:07:42.842326 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:42.842295    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8549799cff-5qgxt"]
Jul 20 21:07:42.961933 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:07:42.961844    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9acc6489_80b6_4a49_b7cc_ff551323d1ec.slice/crio-bbd751f39f5d344c17495bf78c83aadd3417cb2fe527c7d245b6a4df251009b8 WatchSource:0}: Error finding container bbd751f39f5d344c17495bf78c83aadd3417cb2fe527c7d245b6a4df251009b8: Status 404 returned error can't find the container with id bbd751f39f5d344c17495bf78c83aadd3417cb2fe527c7d245b6a4df251009b8
Jul 20 21:07:43.773585 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:43.773500    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40dcb9c0-2c68-4d9e-8f5f-411863c640da","Type":"ContainerStarted","Data":"8064381ce9898b95ad571daf250376034c2eebe0d207401dde19a7119dc208c8"}
Jul 20 21:07:43.773798 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:43.773596    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40dcb9c0-2c68-4d9e-8f5f-411863c640da","Type":"ContainerStarted","Data":"49a530a34e0db9220b6e676b3c3a96e9cc436094e9fdfc7f762eabb4c65d545f"}
Jul 20 21:07:43.773798 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:43.773640    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40dcb9c0-2c68-4d9e-8f5f-411863c640da","Type":"ContainerStarted","Data":"fc9a4f0c121969ff4998062ab1c8441ec4f8f31ab74bd1c8286b97b7ea85c8cb"}
Jul 20 21:07:43.773798 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:43.773683    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40dcb9c0-2c68-4d9e-8f5f-411863c640da","Type":"ContainerStarted","Data":"fd0ecd5b74553559d132ae51314c0ac1d5103c822c9d8452754627a68bbc16de"}
Jul 20 21:07:43.773798 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:43.773722    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40dcb9c0-2c68-4d9e-8f5f-411863c640da","Type":"ContainerStarted","Data":"54269cdbf4daeb678b535006f744a2caae3eac341b21207e6bb0ea5dedd61190"}
Jul 20 21:07:43.775192 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:43.775155    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8549799cff-5qgxt" event={"ID":"9acc6489-80b6-4a49-b7cc-ff551323d1ec","Type":"ContainerStarted","Data":"bbd751f39f5d344c17495bf78c83aadd3417cb2fe527c7d245b6a4df251009b8"}
Jul 20 21:07:45.721236 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:45.721003    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:07:46.799585 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:46.799552    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40dcb9c0-2c68-4d9e-8f5f-411863c640da","Type":"ContainerStarted","Data":"90ee6d516f88707567c0178ec83439559d7286bd8cf76650b27e61f57bcf883a"}
Jul 20 21:07:46.828817 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:46.828761    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=10.420669352 podStartE2EDuration="16.828738903s" totalImagesPullingTime="6.39106686s" totalInitContainerRuntime="17.002691ms" isStatefulPod=false podCreationTimestamp="2026-07-20 21:07:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:07:46.826775927 +0000 UTC m=+307.730765506" watchObservedRunningTime="2026-07-20 21:07:46.828738903 +0000 UTC m=+307.732728483"
Jul 20 21:07:47.805760 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:47.805714    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8549799cff-5qgxt" event={"ID":"9acc6489-80b6-4a49-b7cc-ff551323d1ec","Type":"ContainerStarted","Data":"b0cf5fe8ded890194ae29614dacb7399711e631348d22f58fc80cced63082fb9"}
Jul 20 21:07:47.827400 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:47.827347    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8549799cff-5qgxt" podStartSLOduration=2.967688239 podStartE2EDuration="6.827329243s" totalImagesPullingTime="3.859641004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:07:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:07:47.826413565 +0000 UTC m=+308.730403180" watchObservedRunningTime="2026-07-20 21:07:47.827329243 +0000 UTC m=+308.731318822"
Jul 20 21:07:52.373001 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:52.372964    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:52.373514 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:52.373051    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:52.377848 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:52.377825    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:07:52.829591 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:07:52.829560    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:08:15.918227 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:15.918189    2561 generic.go:356] "Generic (PLEG): container finished" podID="a6f93445-e1d9-4ef1-bbb4-aba4bd103cd6" containerID="410741b47af4bff6e6d138d14a744375914009a77227bcf7ca80cbd3c71dbe17" exitCode=0
Jul 20 21:08:15.918635 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:15.918266    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-p84ql" event={"ID":"a6f93445-e1d9-4ef1-bbb4-aba4bd103cd6","Type":"ContainerDied","Data":"410741b47af4bff6e6d138d14a744375914009a77227bcf7ca80cbd3c71dbe17"}
Jul 20 21:08:15.918635 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:15.918604    2561 scope.go:117] "RemoveContainer" containerID="410741b47af4bff6e6d138d14a744375914009a77227bcf7ca80cbd3c71dbe17"
Jul 20 21:08:16.924144 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:16.924099    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-p84ql" event={"ID":"a6f93445-e1d9-4ef1-bbb4-aba4bd103cd6","Type":"ContainerStarted","Data":"cb2f60d5ac089318abef77e8304823f5639dcec7f131ef69a704ef2a359bddf9"}
Jul 20 21:08:20.938814 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:20.938779    2561 generic.go:356] "Generic (PLEG): container finished" podID="844a855d-9faf-4af8-96f4-1985596720ea" containerID="99ab6e93590694d606b381ab6408c73b23041e3cea762b9edca89021b053fc5b" exitCode=0
Jul 20 21:08:20.939458 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:20.938861    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-ckxwr" event={"ID":"844a855d-9faf-4af8-96f4-1985596720ea","Type":"ContainerDied","Data":"99ab6e93590694d606b381ab6408c73b23041e3cea762b9edca89021b053fc5b"}
Jul 20 21:08:20.939458 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:20.939275    2561 scope.go:117] "RemoveContainer" containerID="99ab6e93590694d606b381ab6408c73b23041e3cea762b9edca89021b053fc5b"
Jul 20 21:08:21.944542 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:21.944509    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-ckxwr" event={"ID":"844a855d-9faf-4af8-96f4-1985596720ea","Type":"ContainerStarted","Data":"d5de0c6ecf8caca103a8298ee88d2f3d1a3f94c0ae42de6d588faf71a4baf3e0"}
Jul 20 21:08:47.105991 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.105910    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bfbc5df8c-gsnwf"]
Jul 20 21:08:47.108106 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.108064    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:47.126010 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.125983    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bfbc5df8c-gsnwf"]
Jul 20 21:08:47.183961 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.183923    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfwr7\" (UniqueName: \"kubernetes.io/projected/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-kube-api-access-dfwr7\") pod \"console-6bfbc5df8c-gsnwf\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") " pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:47.183961 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.183960    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-console-serving-cert\") pod \"console-6bfbc5df8c-gsnwf\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") " pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:47.184190 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.184041    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-console-config\") pod \"console-6bfbc5df8c-gsnwf\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") " pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:47.184190 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.184124    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-service-ca\") pod \"console-6bfbc5df8c-gsnwf\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") " pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:47.184190 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.184151    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-oauth-serving-cert\") pod \"console-6bfbc5df8c-gsnwf\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") " pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:47.184190 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.184171    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-console-oauth-config\") pod \"console-6bfbc5df8c-gsnwf\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") " pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:47.184317 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.184191    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-trusted-ca-bundle\") pod \"console-6bfbc5df8c-gsnwf\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") " pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:47.285406 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.285362    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfwr7\" (UniqueName: \"kubernetes.io/projected/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-kube-api-access-dfwr7\") pod \"console-6bfbc5df8c-gsnwf\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") " pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:47.285406 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.285408    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-console-serving-cert\") pod \"console-6bfbc5df8c-gsnwf\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") " pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:47.285631 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.285450    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-console-config\") pod \"console-6bfbc5df8c-gsnwf\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") " pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:47.285631 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.285503    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-service-ca\") pod \"console-6bfbc5df8c-gsnwf\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") " pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:47.285631 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.285525    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-oauth-serving-cert\") pod \"console-6bfbc5df8c-gsnwf\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") " pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:47.285631 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.285552    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-console-oauth-config\") pod \"console-6bfbc5df8c-gsnwf\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") " pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:47.285631 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.285581    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-trusted-ca-bundle\") pod \"console-6bfbc5df8c-gsnwf\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") " pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:47.286306 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.286281    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-service-ca\") pod \"console-6bfbc5df8c-gsnwf\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") " pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:47.286501 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.286309    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-console-config\") pod \"console-6bfbc5df8c-gsnwf\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") " pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:47.286501 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.286379    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-oauth-serving-cert\") pod \"console-6bfbc5df8c-gsnwf\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") " pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:47.286634 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.286594    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-trusted-ca-bundle\") pod \"console-6bfbc5df8c-gsnwf\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") " pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:47.287996 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.287972    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-console-serving-cert\") pod \"console-6bfbc5df8c-gsnwf\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") " pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:47.288107 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.288056    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-console-oauth-config\") pod \"console-6bfbc5df8c-gsnwf\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") " pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:47.293649 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.293624    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfwr7\" (UniqueName: \"kubernetes.io/projected/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-kube-api-access-dfwr7\") pod \"console-6bfbc5df8c-gsnwf\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") " pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:47.417869 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.417780    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:47.546673 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:47.546643    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bfbc5df8c-gsnwf"]
Jul 20 21:08:47.548404 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:08:47.548370    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode78c13b1_73aa_4cbb_9bd4_2dadf0e4970d.slice/crio-14076af6c8e367b4c5a3ac3a81b65af0bd68b4c22357814291deb903070ec538 WatchSource:0}: Error finding container 14076af6c8e367b4c5a3ac3a81b65af0bd68b4c22357814291deb903070ec538: Status 404 returned error can't find the container with id 14076af6c8e367b4c5a3ac3a81b65af0bd68b4c22357814291deb903070ec538
Jul 20 21:08:48.049846 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:48.049807    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bfbc5df8c-gsnwf" event={"ID":"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d","Type":"ContainerStarted","Data":"412a47780e90f7b9462d0dbf44a1a44342a5052571a8b805e6d99f0d31b808b2"}
Jul 20 21:08:48.050042 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:48.049860    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bfbc5df8c-gsnwf" event={"ID":"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d","Type":"ContainerStarted","Data":"14076af6c8e367b4c5a3ac3a81b65af0bd68b4c22357814291deb903070ec538"}
Jul 20 21:08:48.068767 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:48.068722    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6bfbc5df8c-gsnwf" podStartSLOduration=1.068706394 podStartE2EDuration="1.068706394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:08:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:08:48.06670673 +0000 UTC m=+368.970696355" watchObservedRunningTime="2026-07-20 21:08:48.068706394 +0000 UTC m=+368.972695972"
Jul 20 21:08:49.725616 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:49.725587    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 20 21:08:49.726013 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:49.725874    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="alertmanager" containerID="cri-o://54269cdbf4daeb678b535006f744a2caae3eac341b21207e6bb0ea5dedd61190" gracePeriod=120
Jul 20 21:08:49.726013 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:49.725902    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="prom-label-proxy" containerID="cri-o://90ee6d516f88707567c0178ec83439559d7286bd8cf76650b27e61f57bcf883a" gracePeriod=120
Jul 20 21:08:49.726013 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:49.725919    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="kube-rbac-proxy-metric" containerID="cri-o://8064381ce9898b95ad571daf250376034c2eebe0d207401dde19a7119dc208c8" gracePeriod=120
Jul 20 21:08:49.726013 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:49.725938    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="kube-rbac-proxy-web" containerID="cri-o://fc9a4f0c121969ff4998062ab1c8441ec4f8f31ab74bd1c8286b97b7ea85c8cb" gracePeriod=120
Jul 20 21:08:49.726013 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:49.725952    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="config-reloader" containerID="cri-o://fd0ecd5b74553559d132ae51314c0ac1d5103c822c9d8452754627a68bbc16de" gracePeriod=120
Jul 20 21:08:49.726013 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:49.725946    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="kube-rbac-proxy" containerID="cri-o://49a530a34e0db9220b6e676b3c3a96e9cc436094e9fdfc7f762eabb4c65d545f" gracePeriod=120
Jul 20 21:08:50.060325 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:50.060290    2561 generic.go:356] "Generic (PLEG): container finished" podID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerID="90ee6d516f88707567c0178ec83439559d7286bd8cf76650b27e61f57bcf883a" exitCode=0
Jul 20 21:08:50.060325 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:50.060319    2561 generic.go:356] "Generic (PLEG): container finished" podID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerID="8064381ce9898b95ad571daf250376034c2eebe0d207401dde19a7119dc208c8" exitCode=0
Jul 20 21:08:50.060325 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:50.060326    2561 generic.go:356] "Generic (PLEG): container finished" podID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerID="49a530a34e0db9220b6e676b3c3a96e9cc436094e9fdfc7f762eabb4c65d545f" exitCode=0
Jul 20 21:08:50.060325 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:50.060332    2561 generic.go:356] "Generic (PLEG): container finished" podID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerID="fd0ecd5b74553559d132ae51314c0ac1d5103c822c9d8452754627a68bbc16de" exitCode=0
Jul 20 21:08:50.060567 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:50.060341    2561 generic.go:356] "Generic (PLEG): container finished" podID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerID="54269cdbf4daeb678b535006f744a2caae3eac341b21207e6bb0ea5dedd61190" exitCode=0
Jul 20 21:08:50.060567 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:50.060359    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40dcb9c0-2c68-4d9e-8f5f-411863c640da","Type":"ContainerDied","Data":"90ee6d516f88707567c0178ec83439559d7286bd8cf76650b27e61f57bcf883a"}
Jul 20 21:08:50.060567 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:50.060443    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40dcb9c0-2c68-4d9e-8f5f-411863c640da","Type":"ContainerDied","Data":"8064381ce9898b95ad571daf250376034c2eebe0d207401dde19a7119dc208c8"}
Jul 20 21:08:50.060567 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:50.060468    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40dcb9c0-2c68-4d9e-8f5f-411863c640da","Type":"ContainerDied","Data":"49a530a34e0db9220b6e676b3c3a96e9cc436094e9fdfc7f762eabb4c65d545f"}
Jul 20 21:08:50.060567 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:50.060492    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40dcb9c0-2c68-4d9e-8f5f-411863c640da","Type":"ContainerDied","Data":"fd0ecd5b74553559d132ae51314c0ac1d5103c822c9d8452754627a68bbc16de"}
Jul 20 21:08:50.060567 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:50.060517    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40dcb9c0-2c68-4d9e-8f5f-411863c640da","Type":"ContainerDied","Data":"54269cdbf4daeb678b535006f744a2caae3eac341b21207e6bb0ea5dedd61190"}
Jul 20 21:08:51.069167 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.069122    2561 generic.go:356] "Generic (PLEG): container finished" podID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerID="fc9a4f0c121969ff4998062ab1c8441ec4f8f31ab74bd1c8286b97b7ea85c8cb" exitCode=0
Jul 20 21:08:51.069501 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.069186    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40dcb9c0-2c68-4d9e-8f5f-411863c640da","Type":"ContainerDied","Data":"fc9a4f0c121969ff4998062ab1c8441ec4f8f31ab74bd1c8286b97b7ea85c8cb"}
Jul 20 21:08:51.069501 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.069247    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"40dcb9c0-2c68-4d9e-8f5f-411863c640da","Type":"ContainerDied","Data":"64f9add49610064f1153bd7d5b6f60b9c68de12ee0e36dffc6c76e4da5d99b2b"}
Jul 20 21:08:51.069501 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.069272    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64f9add49610064f1153bd7d5b6f60b9c68de12ee0e36dffc6c76e4da5d99b2b"
Jul 20 21:08:51.077054 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.077034    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:51.119918 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.119844    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-config-volume\") pod \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") "
Jul 20 21:08:51.119918 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.119911    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/40dcb9c0-2c68-4d9e-8f5f-411863c640da-alertmanager-main-db\") pod \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") "
Jul 20 21:08:51.120147 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.119945    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5p8cp\" (UniqueName: \"kubernetes.io/projected/40dcb9c0-2c68-4d9e-8f5f-411863c640da-kube-api-access-5p8cp\") pod \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") "
Jul 20 21:08:51.120147 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.119974    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-main-tls\") pod \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") "
Jul 20 21:08:51.120147 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.120009    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-kube-rbac-proxy-metric\") pod \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") "
Jul 20 21:08:51.120147 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.120047    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40dcb9c0-2c68-4d9e-8f5f-411863c640da-alertmanager-trusted-ca-bundle\") pod \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") "
Jul 20 21:08:51.120147 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.120092    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-web-config\") pod \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") "
Jul 20 21:08:51.120147 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.120137    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40dcb9c0-2c68-4d9e-8f5f-411863c640da-metrics-client-ca\") pod \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") "
Jul 20 21:08:51.120439 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.120171    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-cluster-tls-config\") pod \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") "
Jul 20 21:08:51.120439 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.120232    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/40dcb9c0-2c68-4d9e-8f5f-411863c640da-tls-assets\") pod \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") "
Jul 20 21:08:51.120439 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.120244    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40dcb9c0-2c68-4d9e-8f5f-411863c640da-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "40dcb9c0-2c68-4d9e-8f5f-411863c640da" (UID: "40dcb9c0-2c68-4d9e-8f5f-411863c640da"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:08:51.120439 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.120257    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-kube-rbac-proxy\") pod \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") "
Jul 20 21:08:51.120439 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.120324    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-kube-rbac-proxy-web\") pod \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") "
Jul 20 21:08:51.120439 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.120370    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/40dcb9c0-2c68-4d9e-8f5f-411863c640da-config-out\") pod \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\" (UID: \"40dcb9c0-2c68-4d9e-8f5f-411863c640da\") "
Jul 20 21:08:51.120741 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.120681    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40dcb9c0-2c68-4d9e-8f5f-411863c640da-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "40dcb9c0-2c68-4d9e-8f5f-411863c640da" (UID: "40dcb9c0-2c68-4d9e-8f5f-411863c640da"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 21:08:51.120805 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.120737    2561 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/40dcb9c0-2c68-4d9e-8f5f-411863c640da-alertmanager-main-db\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:08:51.121122 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.121097    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40dcb9c0-2c68-4d9e-8f5f-411863c640da-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "40dcb9c0-2c68-4d9e-8f5f-411863c640da" (UID: "40dcb9c0-2c68-4d9e-8f5f-411863c640da"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 21:08:51.123392 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.123362    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "40dcb9c0-2c68-4d9e-8f5f-411863c640da" (UID: "40dcb9c0-2c68-4d9e-8f5f-411863c640da"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 21:08:51.123851 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.123800    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "40dcb9c0-2c68-4d9e-8f5f-411863c640da" (UID: "40dcb9c0-2c68-4d9e-8f5f-411863c640da"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 21:08:51.124030 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.124003    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40dcb9c0-2c68-4d9e-8f5f-411863c640da-config-out" (OuterVolumeSpecName: "config-out") pod "40dcb9c0-2c68-4d9e-8f5f-411863c640da" (UID: "40dcb9c0-2c68-4d9e-8f5f-411863c640da"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:08:51.124354 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.124319    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40dcb9c0-2c68-4d9e-8f5f-411863c640da-kube-api-access-5p8cp" (OuterVolumeSpecName: "kube-api-access-5p8cp") pod "40dcb9c0-2c68-4d9e-8f5f-411863c640da" (UID: "40dcb9c0-2c68-4d9e-8f5f-411863c640da"). InnerVolumeSpecName "kube-api-access-5p8cp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 21:08:51.124790 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.124765    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-config-volume" (OuterVolumeSpecName: "config-volume") pod "40dcb9c0-2c68-4d9e-8f5f-411863c640da" (UID: "40dcb9c0-2c68-4d9e-8f5f-411863c640da"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 21:08:51.124893 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.124873    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40dcb9c0-2c68-4d9e-8f5f-411863c640da-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "40dcb9c0-2c68-4d9e-8f5f-411863c640da" (UID: "40dcb9c0-2c68-4d9e-8f5f-411863c640da"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 21:08:51.125399 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.125375    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "40dcb9c0-2c68-4d9e-8f5f-411863c640da" (UID: "40dcb9c0-2c68-4d9e-8f5f-411863c640da"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 21:08:51.130417 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.130238    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "40dcb9c0-2c68-4d9e-8f5f-411863c640da" (UID: "40dcb9c0-2c68-4d9e-8f5f-411863c640da"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 21:08:51.132658 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.132621    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "40dcb9c0-2c68-4d9e-8f5f-411863c640da" (UID: "40dcb9c0-2c68-4d9e-8f5f-411863c640da"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 21:08:51.139022 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.138887    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-web-config" (OuterVolumeSpecName: "web-config") pod "40dcb9c0-2c68-4d9e-8f5f-411863c640da" (UID: "40dcb9c0-2c68-4d9e-8f5f-411863c640da"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 21:08:51.221694 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.221654    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5p8cp\" (UniqueName: \"kubernetes.io/projected/40dcb9c0-2c68-4d9e-8f5f-411863c640da-kube-api-access-5p8cp\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:08:51.221694 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.221685    2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-main-tls\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:08:51.221694 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.221695    2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:08:51.221917 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.221705    2561 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40dcb9c0-2c68-4d9e-8f5f-411863c640da-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:08:51.221917 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.221716    2561 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-web-config\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:08:51.221917 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.221725    2561 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40dcb9c0-2c68-4d9e-8f5f-411863c640da-metrics-client-ca\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:08:51.221917 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.221734    2561 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-cluster-tls-config\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:08:51.221917 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.221743    2561 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/40dcb9c0-2c68-4d9e-8f5f-411863c640da-tls-assets\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:08:51.221917 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.221751    2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:08:51.221917 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.221759    2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:08:51.221917 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.221768    2561 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/40dcb9c0-2c68-4d9e-8f5f-411863c640da-config-out\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:08:51.221917 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:51.221776    2561 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/40dcb9c0-2c68-4d9e-8f5f-411863c640da-config-volume\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:08:52.073336 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.073305    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.094310 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.094278    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 20 21:08:52.098463 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.098436    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 20 21:08:52.126534 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.126500    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 20 21:08:52.126839 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.126822    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="kube-rbac-proxy-web"
Jul 20 21:08:52.126907 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.126842    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="kube-rbac-proxy-web"
Jul 20 21:08:52.126907 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.126860    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="config-reloader"
Jul 20 21:08:52.126907 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.126868    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="config-reloader"
Jul 20 21:08:52.126907 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.126886    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="prom-label-proxy"
Jul 20 21:08:52.126907 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.126895    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="prom-label-proxy"
Jul 20 21:08:52.127167 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.126912    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="init-config-reloader"
Jul 20 21:08:52.127167 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.126920    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="init-config-reloader"
Jul 20 21:08:52.127167 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.126930    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="alertmanager"
Jul 20 21:08:52.127167 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.126938    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="alertmanager"
Jul 20 21:08:52.127167 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.126950    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="kube-rbac-proxy"
Jul 20 21:08:52.127167 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.126959    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="kube-rbac-proxy"
Jul 20 21:08:52.127167 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.126977    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="kube-rbac-proxy-metric"
Jul 20 21:08:52.127167 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.127017    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="kube-rbac-proxy-metric"
Jul 20 21:08:52.127167 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.127115    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="kube-rbac-proxy-web"
Jul 20 21:08:52.127167 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.127128    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="kube-rbac-proxy-metric"
Jul 20 21:08:52.127167 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.127138    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="alertmanager"
Jul 20 21:08:52.127167 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.127147    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="config-reloader"
Jul 20 21:08:52.127167 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.127157    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="prom-label-proxy"
Jul 20 21:08:52.127167 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.127168    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" containerName="kube-rbac-proxy"
Jul 20 21:08:52.130410 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.130388    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.133007 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.132989    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 20 21:08:52.133136 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.133118    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 20 21:08:52.133209 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.132989    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 20 21:08:52.133209 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.132989    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 20 21:08:52.133317 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.133210    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-j5llk\""
Jul 20 21:08:52.133317 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.133215    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 20 21:08:52.133471 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.133456    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 20 21:08:52.133504 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.133468    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 20 21:08:52.133548 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.133462    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 20 21:08:52.138834 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.138812    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 20 21:08:52.147736 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.147714    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 20 21:08:52.230537 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.230497    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dab17d8e-c490-4e18-bf39-b44cb28eb8db-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.230537 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.230534    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dab17d8e-c490-4e18-bf39-b44cb28eb8db-config-out\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.230771 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.230552    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dab17d8e-c490-4e18-bf39-b44cb28eb8db-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.230771 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.230583    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dab17d8e-c490-4e18-bf39-b44cb28eb8db-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.230771 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.230603    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dab17d8e-c490-4e18-bf39-b44cb28eb8db-config-volume\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.230771 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.230618    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dab17d8e-c490-4e18-bf39-b44cb28eb8db-web-config\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.230771 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.230645    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dab17d8e-c490-4e18-bf39-b44cb28eb8db-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.230771 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.230661    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dab17d8e-c490-4e18-bf39-b44cb28eb8db-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.230771 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.230680    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lttc9\" (UniqueName: \"kubernetes.io/projected/dab17d8e-c490-4e18-bf39-b44cb28eb8db-kube-api-access-lttc9\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.230771 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.230699    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dab17d8e-c490-4e18-bf39-b44cb28eb8db-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.230771 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.230722    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dab17d8e-c490-4e18-bf39-b44cb28eb8db-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.230771 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.230747    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dab17d8e-c490-4e18-bf39-b44cb28eb8db-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.230771 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.230773    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dab17d8e-c490-4e18-bf39-b44cb28eb8db-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.331244 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.331149    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dab17d8e-c490-4e18-bf39-b44cb28eb8db-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.331244 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.331200    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dab17d8e-c490-4e18-bf39-b44cb28eb8db-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.331244 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.331239    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dab17d8e-c490-4e18-bf39-b44cb28eb8db-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.331510 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.331272    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dab17d8e-c490-4e18-bf39-b44cb28eb8db-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.331510 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.331323    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dab17d8e-c490-4e18-bf39-b44cb28eb8db-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.331510 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.331349    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dab17d8e-c490-4e18-bf39-b44cb28eb8db-config-out\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.331510 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.331370    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dab17d8e-c490-4e18-bf39-b44cb28eb8db-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.331510 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.331403    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dab17d8e-c490-4e18-bf39-b44cb28eb8db-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.331510 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.331438    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dab17d8e-c490-4e18-bf39-b44cb28eb8db-config-volume\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.331510 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.331463    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dab17d8e-c490-4e18-bf39-b44cb28eb8db-web-config\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.331510 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.331501    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dab17d8e-c490-4e18-bf39-b44cb28eb8db-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.331917 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.331523    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dab17d8e-c490-4e18-bf39-b44cb28eb8db-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.331917 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.331545    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lttc9\" (UniqueName: \"kubernetes.io/projected/dab17d8e-c490-4e18-bf39-b44cb28eb8db-kube-api-access-lttc9\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.331917 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.331632    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dab17d8e-c490-4e18-bf39-b44cb28eb8db-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.332558 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.332532    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dab17d8e-c490-4e18-bf39-b44cb28eb8db-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.333553 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.333526    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dab17d8e-c490-4e18-bf39-b44cb28eb8db-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.334669 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.334544    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dab17d8e-c490-4e18-bf39-b44cb28eb8db-config-out\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.334669 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.334562    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dab17d8e-c490-4e18-bf39-b44cb28eb8db-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.334669 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.334583    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dab17d8e-c490-4e18-bf39-b44cb28eb8db-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.334669 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.334622    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dab17d8e-c490-4e18-bf39-b44cb28eb8db-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.334669 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.334671    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dab17d8e-c490-4e18-bf39-b44cb28eb8db-web-config\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.334947 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.334752    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dab17d8e-c490-4e18-bf39-b44cb28eb8db-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.334947 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.334916    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dab17d8e-c490-4e18-bf39-b44cb28eb8db-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.336105 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.336068    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dab17d8e-c490-4e18-bf39-b44cb28eb8db-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.336423 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.336404    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dab17d8e-c490-4e18-bf39-b44cb28eb8db-config-volume\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.339975 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.339956    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lttc9\" (UniqueName: \"kubernetes.io/projected/dab17d8e-c490-4e18-bf39-b44cb28eb8db-kube-api-access-lttc9\") pod \"alertmanager-main-0\" (UID: \"dab17d8e-c490-4e18-bf39-b44cb28eb8db\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.440229 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.440190    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 20 21:08:52.580036 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:52.580007    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 20 21:08:52.581677 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:08:52.581605    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddab17d8e_c490_4e18_bf39_b44cb28eb8db.slice/crio-ff23932f3a0ebe06368d141a73e3f9ebf2c8b893c8ce5315676e0d60379cdc42 WatchSource:0}: Error finding container ff23932f3a0ebe06368d141a73e3f9ebf2c8b893c8ce5315676e0d60379cdc42: Status 404 returned error can't find the container with id ff23932f3a0ebe06368d141a73e3f9ebf2c8b893c8ce5315676e0d60379cdc42
Jul 20 21:08:53.078399 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:53.078365    2561 generic.go:356] "Generic (PLEG): container finished" podID="dab17d8e-c490-4e18-bf39-b44cb28eb8db" containerID="46280f1fafb207080d4883578481b092444d749f4e01dfd9b48eff9e522721f6" exitCode=0
Jul 20 21:08:53.078791 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:53.078453    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dab17d8e-c490-4e18-bf39-b44cb28eb8db","Type":"ContainerDied","Data":"46280f1fafb207080d4883578481b092444d749f4e01dfd9b48eff9e522721f6"}
Jul 20 21:08:53.078791 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:53.078522    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dab17d8e-c490-4e18-bf39-b44cb28eb8db","Type":"ContainerStarted","Data":"ff23932f3a0ebe06368d141a73e3f9ebf2c8b893c8ce5315676e0d60379cdc42"}
Jul 20 21:08:53.440101 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:53.440051    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs\") pod \"network-metrics-daemon-khqfb\" (UID: \"3f6a3926-e94e-4981-9952-3ec7307a1e48\") " pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:08:53.442187 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:53.442169    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f6a3926-e94e-4981-9952-3ec7307a1e48-metrics-certs\") pod \"network-metrics-daemon-khqfb\" (UID: \"3f6a3926-e94e-4981-9952-3ec7307a1e48\") " pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:08:53.524614 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:53.524581    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hpjcj\""
Jul 20 21:08:53.532688 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:53.532665    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khqfb"
Jul 20 21:08:53.663789 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:53.663766    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-khqfb"]
Jul 20 21:08:53.666298 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:08:53.666273    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f6a3926_e94e_4981_9952_3ec7307a1e48.slice/crio-749f1d87d9bda18ef8b130670c76e787108a35fa78c0336f1295bf2115317294 WatchSource:0}: Error finding container 749f1d87d9bda18ef8b130670c76e787108a35fa78c0336f1295bf2115317294: Status 404 returned error can't find the container with id 749f1d87d9bda18ef8b130670c76e787108a35fa78c0336f1295bf2115317294
Jul 20 21:08:53.725852 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:53.725815    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40dcb9c0-2c68-4d9e-8f5f-411863c640da" path="/var/lib/kubelet/pods/40dcb9c0-2c68-4d9e-8f5f-411863c640da/volumes"
Jul 20 21:08:54.086920 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:54.086883    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dab17d8e-c490-4e18-bf39-b44cb28eb8db","Type":"ContainerStarted","Data":"c1f081c8a4d9f4de0b1146c45a7e0f5dc14712415c3eb99c7c3cd266c3d33d5d"}
Jul 20 21:08:54.087381 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:54.086953    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dab17d8e-c490-4e18-bf39-b44cb28eb8db","Type":"ContainerStarted","Data":"aea731ecee034fee63812623b5552af41644148ea6825dbc0c0e8330ad8511bc"}
Jul 20 21:08:54.087381 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:54.086980    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dab17d8e-c490-4e18-bf39-b44cb28eb8db","Type":"ContainerStarted","Data":"00ebf5dd0f6ea256cb4c7e9bf9699147f9bf5b8f1b59c934bdf3370ed48425ed"}
Jul 20 21:08:54.087381 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:54.087019    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dab17d8e-c490-4e18-bf39-b44cb28eb8db","Type":"ContainerStarted","Data":"30e1ee6a0f2032ad77d0c6f2a5996c08c0c743c357927812aa616f9382f15287"}
Jul 20 21:08:54.087381 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:54.087059    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dab17d8e-c490-4e18-bf39-b44cb28eb8db","Type":"ContainerStarted","Data":"da3a04dbba35760a08ada57e71c27876274de611b1045ecd12e597039cbf3df4"}
Jul 20 21:08:54.087381 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:54.087115    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dab17d8e-c490-4e18-bf39-b44cb28eb8db","Type":"ContainerStarted","Data":"40351ee6cde15c303907a4ae2d1da75e5ec0718a4a62ffc64838849412dde363"}
Jul 20 21:08:54.088123 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:54.088096    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-khqfb" event={"ID":"3f6a3926-e94e-4981-9952-3ec7307a1e48","Type":"ContainerStarted","Data":"749f1d87d9bda18ef8b130670c76e787108a35fa78c0336f1295bf2115317294"}
Jul 20 21:08:54.119798 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:54.119747    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.102889101 podStartE2EDuration="2.119728433s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.839332ms" isStatefulPod=false podCreationTimestamp="2026-07-20 21:08:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:08:54.118422576 +0000 UTC m=+375.022412155" watchObservedRunningTime="2026-07-20 21:08:54.119728433 +0000 UTC m=+375.023718011"
Jul 20 21:08:55.094756 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:55.094664    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-khqfb" event={"ID":"3f6a3926-e94e-4981-9952-3ec7307a1e48","Type":"ContainerStarted","Data":"71ed8a381cbe6e90360410f7cffe370b88ff77348f3efd8e8fa44be7ac89e669"}
Jul 20 21:08:55.094756 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:55.094730    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-khqfb" event={"ID":"3f6a3926-e94e-4981-9952-3ec7307a1e48","Type":"ContainerStarted","Data":"18bc5d9cf0daa734b7e8c2b6831e79afbb21cbf8d554877da906d8d2a3071578"}
Jul 20 21:08:55.115559 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:55.115504    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-khqfb" podStartSLOduration=374.994373457 podStartE2EDuration="6m16.115488365s" totalImagesPullingTime="1.121114908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:08:55.114397458 +0000 UTC m=+376.018387038" watchObservedRunningTime="2026-07-20 21:08:55.115488365 +0000 UTC m=+376.019477939"
Jul 20 21:08:57.418297 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:57.418262    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:57.418727 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:57.418383    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:57.423222 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:57.423200    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:58.113426 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:58.113398    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:08:58.163196 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:08:58.163164    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8549799cff-5qgxt"]
Jul 20 21:09:23.183310 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:23.183244    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8549799cff-5qgxt" podUID="9acc6489-80b6-4a49-b7cc-ff551323d1ec" containerName="console" containerID="cri-o://b0cf5fe8ded890194ae29614dacb7399711e631348d22f58fc80cced63082fb9" gracePeriod=15
Jul 20 21:09:23.431955 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:23.431930    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:09:23.589975 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:23.589927    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9acc6489-80b6-4a49-b7cc-ff551323d1ec-oauth-serving-cert\") pod \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") "
Jul 20 21:09:23.590212 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:23.590002    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9acc6489-80b6-4a49-b7cc-ff551323d1ec-console-config\") pod \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") "
Jul 20 21:09:23.590212 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:23.590038    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9acc6489-80b6-4a49-b7cc-ff551323d1ec-service-ca\") pod \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") "
Jul 20 21:09:23.590212 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:23.590139    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9acc6489-80b6-4a49-b7cc-ff551323d1ec-console-oauth-config\") pod \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") "
Jul 20 21:09:23.590212 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:23.590163    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9acc6489-80b6-4a49-b7cc-ff551323d1ec-trusted-ca-bundle\") pod \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") "
Jul 20 21:09:23.590212 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:23.590200    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hlfm\" (UniqueName: \"kubernetes.io/projected/9acc6489-80b6-4a49-b7cc-ff551323d1ec-kube-api-access-4hlfm\") pod \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") "
Jul 20 21:09:23.590457 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:23.590224    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9acc6489-80b6-4a49-b7cc-ff551323d1ec-console-serving-cert\") pod \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\" (UID: \"9acc6489-80b6-4a49-b7cc-ff551323d1ec\") "
Jul 20 21:09:23.590508 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:23.590457    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9acc6489-80b6-4a49-b7cc-ff551323d1ec-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9acc6489-80b6-4a49-b7cc-ff551323d1ec" (UID: "9acc6489-80b6-4a49-b7cc-ff551323d1ec"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 21:09:23.590566 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:23.590532    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9acc6489-80b6-4a49-b7cc-ff551323d1ec-console-config" (OuterVolumeSpecName: "console-config") pod "9acc6489-80b6-4a49-b7cc-ff551323d1ec" (UID: "9acc6489-80b6-4a49-b7cc-ff551323d1ec"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 21:09:23.590644 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:23.590621    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9acc6489-80b6-4a49-b7cc-ff551323d1ec-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9acc6489-80b6-4a49-b7cc-ff551323d1ec" (UID: "9acc6489-80b6-4a49-b7cc-ff551323d1ec"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 21:09:23.590708 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:23.590622    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9acc6489-80b6-4a49-b7cc-ff551323d1ec-service-ca" (OuterVolumeSpecName: "service-ca") pod "9acc6489-80b6-4a49-b7cc-ff551323d1ec" (UID: "9acc6489-80b6-4a49-b7cc-ff551323d1ec"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 21:09:23.592508 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:23.592478    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9acc6489-80b6-4a49-b7cc-ff551323d1ec-kube-api-access-4hlfm" (OuterVolumeSpecName: "kube-api-access-4hlfm") pod "9acc6489-80b6-4a49-b7cc-ff551323d1ec" (UID: "9acc6489-80b6-4a49-b7cc-ff551323d1ec"). InnerVolumeSpecName "kube-api-access-4hlfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 21:09:23.592625 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:23.592510    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9acc6489-80b6-4a49-b7cc-ff551323d1ec-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9acc6489-80b6-4a49-b7cc-ff551323d1ec" (UID: "9acc6489-80b6-4a49-b7cc-ff551323d1ec"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 21:09:23.592625 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:23.592549    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9acc6489-80b6-4a49-b7cc-ff551323d1ec-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9acc6489-80b6-4a49-b7cc-ff551323d1ec" (UID: "9acc6489-80b6-4a49-b7cc-ff551323d1ec"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 21:09:23.691874 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:23.691826    2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9acc6489-80b6-4a49-b7cc-ff551323d1ec-oauth-serving-cert\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:09:23.691874 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:23.691869    2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9acc6489-80b6-4a49-b7cc-ff551323d1ec-console-config\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:09:23.691874 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:23.691879    2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9acc6489-80b6-4a49-b7cc-ff551323d1ec-service-ca\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:09:23.691874 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:23.691892    2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9acc6489-80b6-4a49-b7cc-ff551323d1ec-console-oauth-config\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:09:23.692185 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:23.691901    2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9acc6489-80b6-4a49-b7cc-ff551323d1ec-trusted-ca-bundle\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:09:23.692185 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:23.691910    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hlfm\" (UniqueName: \"kubernetes.io/projected/9acc6489-80b6-4a49-b7cc-ff551323d1ec-kube-api-access-4hlfm\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:09:23.692185 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:23.691918    2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9acc6489-80b6-4a49-b7cc-ff551323d1ec-console-serving-cert\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:09:24.214582 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:24.214546    2561 generic.go:356] "Generic (PLEG): container finished" podID="9acc6489-80b6-4a49-b7cc-ff551323d1ec" containerID="b0cf5fe8ded890194ae29614dacb7399711e631348d22f58fc80cced63082fb9" exitCode=0
Jul 20 21:09:24.215108 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:24.214596    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8549799cff-5qgxt"
Jul 20 21:09:24.215108 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:24.214630    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8549799cff-5qgxt" event={"ID":"9acc6489-80b6-4a49-b7cc-ff551323d1ec","Type":"ContainerDied","Data":"b0cf5fe8ded890194ae29614dacb7399711e631348d22f58fc80cced63082fb9"}
Jul 20 21:09:24.215108 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:24.214684    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8549799cff-5qgxt" event={"ID":"9acc6489-80b6-4a49-b7cc-ff551323d1ec","Type":"ContainerDied","Data":"bbd751f39f5d344c17495bf78c83aadd3417cb2fe527c7d245b6a4df251009b8"}
Jul 20 21:09:24.215108 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:24.214706    2561 scope.go:117] "RemoveContainer" containerID="b0cf5fe8ded890194ae29614dacb7399711e631348d22f58fc80cced63082fb9"
Jul 20 21:09:24.222847 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:24.222833    2561 scope.go:117] "RemoveContainer" containerID="b0cf5fe8ded890194ae29614dacb7399711e631348d22f58fc80cced63082fb9"
Jul 20 21:09:24.223152 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:09:24.223130    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0cf5fe8ded890194ae29614dacb7399711e631348d22f58fc80cced63082fb9\": container with ID starting with b0cf5fe8ded890194ae29614dacb7399711e631348d22f58fc80cced63082fb9 not found: ID does not exist" containerID="b0cf5fe8ded890194ae29614dacb7399711e631348d22f58fc80cced63082fb9"
Jul 20 21:09:24.223230 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:24.223162    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0cf5fe8ded890194ae29614dacb7399711e631348d22f58fc80cced63082fb9"} err="failed to get container status \"b0cf5fe8ded890194ae29614dacb7399711e631348d22f58fc80cced63082fb9\": rpc error: code = NotFound desc = could not find container \"b0cf5fe8ded890194ae29614dacb7399711e631348d22f58fc80cced63082fb9\": container with ID starting with b0cf5fe8ded890194ae29614dacb7399711e631348d22f58fc80cced63082fb9 not found: ID does not exist"
Jul 20 21:09:24.236849 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:24.236825    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8549799cff-5qgxt"]
Jul 20 21:09:24.242188 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:24.242165    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8549799cff-5qgxt"]
Jul 20 21:09:25.724893 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:25.724858    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9acc6489-80b6-4a49-b7cc-ff551323d1ec" path="/var/lib/kubelet/pods/9acc6489-80b6-4a49-b7cc-ff551323d1ec/volumes"
Jul 20 21:09:45.872897 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:45.872864    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv"]
Jul 20 21:09:45.873371 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:45.873177    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9acc6489-80b6-4a49-b7cc-ff551323d1ec" containerName="console"
Jul 20 21:09:45.873371 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:45.873189    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9acc6489-80b6-4a49-b7cc-ff551323d1ec" containerName="console"
Jul 20 21:09:45.873371 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:45.873253    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9acc6489-80b6-4a49-b7cc-ff551323d1ec" containerName="console"
Jul 20 21:09:45.877667 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:45.877650    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv"
Jul 20 21:09:45.880321 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:45.880300    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 20 21:09:45.880439 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:45.880322    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 20 21:09:45.880439 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:45.880301    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rxvs8\""
Jul 20 21:09:45.888016 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:45.887996    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv"]
Jul 20 21:09:45.984461 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:45.984422    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ad3de5b-f722-4c40-840f-07c4600ac1c7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv\" (UID: \"6ad3de5b-f722-4c40-840f-07c4600ac1c7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv"
Jul 20 21:09:45.984461 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:45.984468    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ad3de5b-f722-4c40-840f-07c4600ac1c7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv\" (UID: \"6ad3de5b-f722-4c40-840f-07c4600ac1c7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv"
Jul 20 21:09:45.984680 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:45.984485    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lljft\" (UniqueName: \"kubernetes.io/projected/6ad3de5b-f722-4c40-840f-07c4600ac1c7-kube-api-access-lljft\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv\" (UID: \"6ad3de5b-f722-4c40-840f-07c4600ac1c7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv"
Jul 20 21:09:46.085443 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:46.085404    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ad3de5b-f722-4c40-840f-07c4600ac1c7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv\" (UID: \"6ad3de5b-f722-4c40-840f-07c4600ac1c7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv"
Jul 20 21:09:46.085651 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:46.085460    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ad3de5b-f722-4c40-840f-07c4600ac1c7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv\" (UID: \"6ad3de5b-f722-4c40-840f-07c4600ac1c7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv"
Jul 20 21:09:46.085651 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:46.085486    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lljft\" (UniqueName: \"kubernetes.io/projected/6ad3de5b-f722-4c40-840f-07c4600ac1c7-kube-api-access-lljft\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv\" (UID: \"6ad3de5b-f722-4c40-840f-07c4600ac1c7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv"
Jul 20 21:09:46.085811 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:46.085789    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ad3de5b-f722-4c40-840f-07c4600ac1c7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv\" (UID: \"6ad3de5b-f722-4c40-840f-07c4600ac1c7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv"
Jul 20 21:09:46.085871 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:46.085823    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ad3de5b-f722-4c40-840f-07c4600ac1c7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv\" (UID: \"6ad3de5b-f722-4c40-840f-07c4600ac1c7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv"
Jul 20 21:09:46.095418 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:46.095391    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lljft\" (UniqueName: \"kubernetes.io/projected/6ad3de5b-f722-4c40-840f-07c4600ac1c7-kube-api-access-lljft\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv\" (UID: \"6ad3de5b-f722-4c40-840f-07c4600ac1c7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv"
Jul 20 21:09:46.186312 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:46.186218    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv"
Jul 20 21:09:46.315101 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:46.315060    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv"]
Jul 20 21:09:46.317478 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:09:46.317446    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ad3de5b_f722_4c40_840f_07c4600ac1c7.slice/crio-7b7cbc9785f640c99638ea3a6c83eb27249efb005b7497bd77a675f3813f6ece WatchSource:0}: Error finding container 7b7cbc9785f640c99638ea3a6c83eb27249efb005b7497bd77a675f3813f6ece: Status 404 returned error can't find the container with id 7b7cbc9785f640c99638ea3a6c83eb27249efb005b7497bd77a675f3813f6ece
Jul 20 21:09:47.307207 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:47.307172    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv" event={"ID":"6ad3de5b-f722-4c40-840f-07c4600ac1c7","Type":"ContainerStarted","Data":"7b7cbc9785f640c99638ea3a6c83eb27249efb005b7497bd77a675f3813f6ece"}
Jul 20 21:09:53.333372 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:53.333338    2561 generic.go:356] "Generic (PLEG): container finished" podID="6ad3de5b-f722-4c40-840f-07c4600ac1c7" containerID="1e39a426a81b1e83584466f8573386d06bfa4325751d9ad151ca831d778be9ae" exitCode=0
Jul 20 21:09:53.333695 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:53.333412    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv" event={"ID":"6ad3de5b-f722-4c40-840f-07c4600ac1c7","Type":"ContainerDied","Data":"1e39a426a81b1e83584466f8573386d06bfa4325751d9ad151ca831d778be9ae"}
Jul 20 21:09:56.347871 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:56.347835    2561 generic.go:356] "Generic (PLEG): container finished" podID="6ad3de5b-f722-4c40-840f-07c4600ac1c7" containerID="ffb8665c6499a866491926d3c2f71ad314cd08d4860fc7bbf23ebc6192bde3a5" exitCode=0
Jul 20 21:09:56.347871 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:09:56.347875    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv" event={"ID":"6ad3de5b-f722-4c40-840f-07c4600ac1c7","Type":"ContainerDied","Data":"ffb8665c6499a866491926d3c2f71ad314cd08d4860fc7bbf23ebc6192bde3a5"}
Jul 20 21:10:04.385382 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:04.385342    2561 generic.go:356] "Generic (PLEG): container finished" podID="6ad3de5b-f722-4c40-840f-07c4600ac1c7" containerID="cb1a048b202a2e2a1ab07e871f3fd9f1f362c1672cf5e51abf26e7cd0652a78b" exitCode=0
Jul 20 21:10:04.385762 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:04.385394    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv" event={"ID":"6ad3de5b-f722-4c40-840f-07c4600ac1c7","Type":"ContainerDied","Data":"cb1a048b202a2e2a1ab07e871f3fd9f1f362c1672cf5e51abf26e7cd0652a78b"}
Jul 20 21:10:05.559383 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:05.559349    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv"
Jul 20 21:10:05.659058 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:05.658959    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ad3de5b-f722-4c40-840f-07c4600ac1c7-util\") pod \"6ad3de5b-f722-4c40-840f-07c4600ac1c7\" (UID: \"6ad3de5b-f722-4c40-840f-07c4600ac1c7\") "
Jul 20 21:10:05.659058 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:05.659021    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lljft\" (UniqueName: \"kubernetes.io/projected/6ad3de5b-f722-4c40-840f-07c4600ac1c7-kube-api-access-lljft\") pod \"6ad3de5b-f722-4c40-840f-07c4600ac1c7\" (UID: \"6ad3de5b-f722-4c40-840f-07c4600ac1c7\") "
Jul 20 21:10:05.659313 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:05.659230    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ad3de5b-f722-4c40-840f-07c4600ac1c7-bundle\") pod \"6ad3de5b-f722-4c40-840f-07c4600ac1c7\" (UID: \"6ad3de5b-f722-4c40-840f-07c4600ac1c7\") "
Jul 20 21:10:05.659840 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:05.659814    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ad3de5b-f722-4c40-840f-07c4600ac1c7-bundle" (OuterVolumeSpecName: "bundle") pod "6ad3de5b-f722-4c40-840f-07c4600ac1c7" (UID: "6ad3de5b-f722-4c40-840f-07c4600ac1c7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:10:05.661212 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:05.661190    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ad3de5b-f722-4c40-840f-07c4600ac1c7-kube-api-access-lljft" (OuterVolumeSpecName: "kube-api-access-lljft") pod "6ad3de5b-f722-4c40-840f-07c4600ac1c7" (UID: "6ad3de5b-f722-4c40-840f-07c4600ac1c7"). InnerVolumeSpecName "kube-api-access-lljft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 21:10:05.664564 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:05.664524    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ad3de5b-f722-4c40-840f-07c4600ac1c7-util" (OuterVolumeSpecName: "util") pod "6ad3de5b-f722-4c40-840f-07c4600ac1c7" (UID: "6ad3de5b-f722-4c40-840f-07c4600ac1c7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:10:05.760205 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:05.760171    2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ad3de5b-f722-4c40-840f-07c4600ac1c7-bundle\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:10:05.760205 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:05.760204    2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ad3de5b-f722-4c40-840f-07c4600ac1c7-util\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:10:05.760447 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:05.760217    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lljft\" (UniqueName: \"kubernetes.io/projected/6ad3de5b-f722-4c40-840f-07c4600ac1c7-kube-api-access-lljft\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:10:06.395292 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:06.395254    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv" event={"ID":"6ad3de5b-f722-4c40-840f-07c4600ac1c7","Type":"ContainerDied","Data":"7b7cbc9785f640c99638ea3a6c83eb27249efb005b7497bd77a675f3813f6ece"}
Jul 20 21:10:06.395292 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:06.395288    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m6rzv"
Jul 20 21:10:06.395544 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:06.395305    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7b7cbc9785f640c99638ea3a6c83eb27249efb005b7497bd77a675f3813f6ece"
Jul 20 21:10:24.006901 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:24.006867    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tfg6v"]
Jul 20 21:10:24.007327 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:24.007217    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ad3de5b-f722-4c40-840f-07c4600ac1c7" containerName="pull"
Jul 20 21:10:24.007327 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:24.007229    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ad3de5b-f722-4c40-840f-07c4600ac1c7" containerName="pull"
Jul 20 21:10:24.007327 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:24.007249    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ad3de5b-f722-4c40-840f-07c4600ac1c7" containerName="extract"
Jul 20 21:10:24.007327 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:24.007258    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ad3de5b-f722-4c40-840f-07c4600ac1c7" containerName="extract"
Jul 20 21:10:24.007327 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:24.007272    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ad3de5b-f722-4c40-840f-07c4600ac1c7" containerName="util"
Jul 20 21:10:24.007327 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:24.007276    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ad3de5b-f722-4c40-840f-07c4600ac1c7" containerName="util"
Jul 20 21:10:24.007537 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:24.007332    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ad3de5b-f722-4c40-840f-07c4600ac1c7" containerName="extract"
Jul 20 21:10:24.009471 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:24.009452    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-tfg6v"
Jul 20 21:10:24.011742 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:24.011717    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 20 21:10:24.011858 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:24.011717    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-h678m\""
Jul 20 21:10:24.012690 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:24.012677    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 20 21:10:24.018568 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:24.018549    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tfg6v"]
Jul 20 21:10:24.118779 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:24.118738    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/be75e75a-0343-474c-bd1c-81ae1e90dd87-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tfg6v\" (UID: \"be75e75a-0343-474c-bd1c-81ae1e90dd87\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tfg6v"
Jul 20 21:10:24.118953 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:24.118796    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fn8l\" (UniqueName: \"kubernetes.io/projected/be75e75a-0343-474c-bd1c-81ae1e90dd87-kube-api-access-9fn8l\") pod \"cert-manager-webhook-6486ffd997-tfg6v\" (UID: \"be75e75a-0343-474c-bd1c-81ae1e90dd87\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tfg6v"
Jul 20 21:10:24.219719 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:24.219680    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/be75e75a-0343-474c-bd1c-81ae1e90dd87-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tfg6v\" (UID: \"be75e75a-0343-474c-bd1c-81ae1e90dd87\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tfg6v"
Jul 20 21:10:24.219927 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:24.219739    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fn8l\" (UniqueName: \"kubernetes.io/projected/be75e75a-0343-474c-bd1c-81ae1e90dd87-kube-api-access-9fn8l\") pod \"cert-manager-webhook-6486ffd997-tfg6v\" (UID: \"be75e75a-0343-474c-bd1c-81ae1e90dd87\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tfg6v"
Jul 20 21:10:24.228316 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:24.228289    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fn8l\" (UniqueName: \"kubernetes.io/projected/be75e75a-0343-474c-bd1c-81ae1e90dd87-kube-api-access-9fn8l\") pod \"cert-manager-webhook-6486ffd997-tfg6v\" (UID: \"be75e75a-0343-474c-bd1c-81ae1e90dd87\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tfg6v"
Jul 20 21:10:24.228440 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:24.228321    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/be75e75a-0343-474c-bd1c-81ae1e90dd87-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tfg6v\" (UID: \"be75e75a-0343-474c-bd1c-81ae1e90dd87\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tfg6v"
Jul 20 21:10:24.334221 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:24.334183    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-tfg6v"
Jul 20 21:10:24.462511 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:24.462489    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tfg6v"]
Jul 20 21:10:25.471679 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:25.471635    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-tfg6v" event={"ID":"be75e75a-0343-474c-bd1c-81ae1e90dd87","Type":"ContainerStarted","Data":"6441f5da4508a2afb3c66073d76dd71fe2443f3ad53ad71f9b08c2b2fd49f981"}
Jul 20 21:10:27.482754 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:27.482707    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-tfg6v" event={"ID":"be75e75a-0343-474c-bd1c-81ae1e90dd87","Type":"ContainerStarted","Data":"c3d2a71cbada837f911d544d1c1c42247302d08b5006f7bed560eacee280cfb5"}
Jul 20 21:10:27.483190 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:27.482781    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-tfg6v"
Jul 20 21:10:27.500918 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:27.500869    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-tfg6v" podStartSLOduration=2.021526692 podStartE2EDuration="4.500853668s" totalImagesPullingTime="2.479326976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:10:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:10:27.499850893 +0000 UTC m=+468.403840472" watchObservedRunningTime="2026-07-20 21:10:27.500853668 +0000 UTC m=+468.404843246"
Jul 20 21:10:33.489514 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:33.489478    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-tfg6v"
Jul 20 21:10:50.167293 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:50.167258    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4"]
Jul 20 21:10:50.169727 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:50.169711    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4"
Jul 20 21:10:50.174319 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:50.174297    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rxvs8\""
Jul 20 21:10:50.174827 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:50.174806    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 20 21:10:50.175243 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:50.175225    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 20 21:10:50.182004 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:50.181982    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4"]
Jul 20 21:10:50.227065 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:50.227026    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96426060-68c7-4c4e-bbb7-70444e557af4-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4\" (UID: \"96426060-68c7-4c4e-bbb7-70444e557af4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4"
Jul 20 21:10:50.227245 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:50.227112    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96426060-68c7-4c4e-bbb7-70444e557af4-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4\" (UID: \"96426060-68c7-4c4e-bbb7-70444e557af4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4"
Jul 20 21:10:50.227245 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:50.227160    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49lg9\" (UniqueName: \"kubernetes.io/projected/96426060-68c7-4c4e-bbb7-70444e557af4-kube-api-access-49lg9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4\" (UID: \"96426060-68c7-4c4e-bbb7-70444e557af4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4"
Jul 20 21:10:50.327880 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:50.327823    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49lg9\" (UniqueName: \"kubernetes.io/projected/96426060-68c7-4c4e-bbb7-70444e557af4-kube-api-access-49lg9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4\" (UID: \"96426060-68c7-4c4e-bbb7-70444e557af4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4"
Jul 20 21:10:50.327880 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:50.327887    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96426060-68c7-4c4e-bbb7-70444e557af4-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4\" (UID: \"96426060-68c7-4c4e-bbb7-70444e557af4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4"
Jul 20 21:10:50.328181 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:50.327949    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96426060-68c7-4c4e-bbb7-70444e557af4-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4\" (UID: \"96426060-68c7-4c4e-bbb7-70444e557af4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4"
Jul 20 21:10:50.328388 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:50.328368    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96426060-68c7-4c4e-bbb7-70444e557af4-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4\" (UID: \"96426060-68c7-4c4e-bbb7-70444e557af4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4"
Jul 20 21:10:50.328501 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:50.328400    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96426060-68c7-4c4e-bbb7-70444e557af4-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4\" (UID: \"96426060-68c7-4c4e-bbb7-70444e557af4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4"
Jul 20 21:10:50.337980 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:50.337957    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49lg9\" (UniqueName: \"kubernetes.io/projected/96426060-68c7-4c4e-bbb7-70444e557af4-kube-api-access-49lg9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4\" (UID: \"96426060-68c7-4c4e-bbb7-70444e557af4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4"
Jul 20 21:10:50.479254 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:50.479173    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4"
Jul 20 21:10:50.609779 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:50.609753    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4"]
Jul 20 21:10:50.612263 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:10:50.612234    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96426060_68c7_4c4e_bbb7_70444e557af4.slice/crio-865691dec98729a33d2c2633b2cd2f5984b7bcbf9e6ced6e20fed0e980f1924f WatchSource:0}: Error finding container 865691dec98729a33d2c2633b2cd2f5984b7bcbf9e6ced6e20fed0e980f1924f: Status 404 returned error can't find the container with id 865691dec98729a33d2c2633b2cd2f5984b7bcbf9e6ced6e20fed0e980f1924f
Jul 20 21:10:51.580695 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:51.580661    2561 generic.go:356] "Generic (PLEG): container finished" podID="96426060-68c7-4c4e-bbb7-70444e557af4" containerID="df132d53a21fa14a0f94ef604a123c8548073c26691201d184b12976f8d72582" exitCode=0
Jul 20 21:10:51.581065 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:51.580743    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4" event={"ID":"96426060-68c7-4c4e-bbb7-70444e557af4","Type":"ContainerDied","Data":"df132d53a21fa14a0f94ef604a123c8548073c26691201d184b12976f8d72582"}
Jul 20 21:10:51.581065 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:51.580798    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4" event={"ID":"96426060-68c7-4c4e-bbb7-70444e557af4","Type":"ContainerStarted","Data":"865691dec98729a33d2c2633b2cd2f5984b7bcbf9e6ced6e20fed0e980f1924f"}
Jul 20 21:10:52.586946 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:52.586906    2561 generic.go:356] "Generic (PLEG): container finished" podID="96426060-68c7-4c4e-bbb7-70444e557af4" containerID="ee570b1d71f88986e1d127b2852c378e0e8051a2cb296fd9d612fb5c7b1317a6" exitCode=0
Jul 20 21:10:52.587405 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:52.586997    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4" event={"ID":"96426060-68c7-4c4e-bbb7-70444e557af4","Type":"ContainerDied","Data":"ee570b1d71f88986e1d127b2852c378e0e8051a2cb296fd9d612fb5c7b1317a6"}
Jul 20 21:10:53.593788 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:53.593748    2561 generic.go:356] "Generic (PLEG): container finished" podID="96426060-68c7-4c4e-bbb7-70444e557af4" containerID="6cafc22b8a27e41bcdd4e21c1b82b13206039be478e1a00e9c10b9fc644ca5f0" exitCode=0
Jul 20 21:10:53.594189 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:53.593836    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4" event={"ID":"96426060-68c7-4c4e-bbb7-70444e557af4","Type":"ContainerDied","Data":"6cafc22b8a27e41bcdd4e21c1b82b13206039be478e1a00e9c10b9fc644ca5f0"}
Jul 20 21:10:54.722142 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:54.722119    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4"
Jul 20 21:10:54.864151 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:54.864045    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96426060-68c7-4c4e-bbb7-70444e557af4-util\") pod \"96426060-68c7-4c4e-bbb7-70444e557af4\" (UID: \"96426060-68c7-4c4e-bbb7-70444e557af4\") "
Jul 20 21:10:54.864151 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:54.864118    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96426060-68c7-4c4e-bbb7-70444e557af4-bundle\") pod \"96426060-68c7-4c4e-bbb7-70444e557af4\" (UID: \"96426060-68c7-4c4e-bbb7-70444e557af4\") "
Jul 20 21:10:54.864379 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:54.864182    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49lg9\" (UniqueName: \"kubernetes.io/projected/96426060-68c7-4c4e-bbb7-70444e557af4-kube-api-access-49lg9\") pod \"96426060-68c7-4c4e-bbb7-70444e557af4\" (UID: \"96426060-68c7-4c4e-bbb7-70444e557af4\") "
Jul 20 21:10:54.864867 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:54.864831    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96426060-68c7-4c4e-bbb7-70444e557af4-bundle" (OuterVolumeSpecName: "bundle") pod "96426060-68c7-4c4e-bbb7-70444e557af4" (UID: "96426060-68c7-4c4e-bbb7-70444e557af4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:10:54.866299 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:54.866269    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96426060-68c7-4c4e-bbb7-70444e557af4-kube-api-access-49lg9" (OuterVolumeSpecName: "kube-api-access-49lg9") pod "96426060-68c7-4c4e-bbb7-70444e557af4" (UID: "96426060-68c7-4c4e-bbb7-70444e557af4"). InnerVolumeSpecName "kube-api-access-49lg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 21:10:54.869727 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:54.869695    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96426060-68c7-4c4e-bbb7-70444e557af4-util" (OuterVolumeSpecName: "util") pod "96426060-68c7-4c4e-bbb7-70444e557af4" (UID: "96426060-68c7-4c4e-bbb7-70444e557af4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:10:54.965692 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:54.965650    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49lg9\" (UniqueName: \"kubernetes.io/projected/96426060-68c7-4c4e-bbb7-70444e557af4-kube-api-access-49lg9\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:10:54.965692 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:54.965687    2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/96426060-68c7-4c4e-bbb7-70444e557af4-util\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:10:54.965692 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:54.965696    2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/96426060-68c7-4c4e-bbb7-70444e557af4-bundle\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:10:55.604728 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:55.604699    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4"
Jul 20 21:10:55.604728 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:55.604715    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg5pl4" event={"ID":"96426060-68c7-4c4e-bbb7-70444e557af4","Type":"ContainerDied","Data":"865691dec98729a33d2c2633b2cd2f5984b7bcbf9e6ced6e20fed0e980f1924f"}
Jul 20 21:10:55.604932 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:10:55.604755    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="865691dec98729a33d2c2633b2cd2f5984b7bcbf9e6ced6e20fed0e980f1924f"
Jul 20 21:11:06.265249 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.265213    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7475cd49b4-wclxq"]
Jul 20 21:11:06.265754 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.265645    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96426060-68c7-4c4e-bbb7-70444e557af4" containerName="pull"
Jul 20 21:11:06.265754 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.265664    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="96426060-68c7-4c4e-bbb7-70444e557af4" containerName="pull"
Jul 20 21:11:06.265754 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.265678    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96426060-68c7-4c4e-bbb7-70444e557af4" containerName="util"
Jul 20 21:11:06.265754 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.265687    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="96426060-68c7-4c4e-bbb7-70444e557af4" containerName="util"
Jul 20 21:11:06.265754 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.265699    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96426060-68c7-4c4e-bbb7-70444e557af4" containerName="extract"
Jul 20 21:11:06.265754 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.265708    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="96426060-68c7-4c4e-bbb7-70444e557af4" containerName="extract"
Jul 20 21:11:06.266051 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.265778    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="96426060-68c7-4c4e-bbb7-70444e557af4" containerName="extract"
Jul 20 21:11:06.268613 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.268594    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7475cd49b4-wclxq"
Jul 20 21:11:06.272870 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.272845    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 20 21:11:06.272970 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.272850    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 20 21:11:06.272970 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.272903    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 20 21:11:06.272970 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.272956    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 20 21:11:06.273128 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.272891    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 20 21:11:06.273128 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.273011    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-vkcvw\""
Jul 20 21:11:06.280437 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.280415    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7475cd49b4-wclxq"]
Jul 20 21:11:06.359195 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.359162    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fc7003b6-c170-420a-ba0a-d94cc9b56c36-manager-config\") pod \"lws-controller-manager-7475cd49b4-wclxq\" (UID: \"fc7003b6-c170-420a-ba0a-d94cc9b56c36\") " pod="openshift-lws-operator/lws-controller-manager-7475cd49b4-wclxq"
Jul 20 21:11:06.359195 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.359199    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc7003b6-c170-420a-ba0a-d94cc9b56c36-cert\") pod \"lws-controller-manager-7475cd49b4-wclxq\" (UID: \"fc7003b6-c170-420a-ba0a-d94cc9b56c36\") " pod="openshift-lws-operator/lws-controller-manager-7475cd49b4-wclxq"
Jul 20 21:11:06.359425 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.359250    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9shp\" (UniqueName: \"kubernetes.io/projected/fc7003b6-c170-420a-ba0a-d94cc9b56c36-kube-api-access-f9shp\") pod \"lws-controller-manager-7475cd49b4-wclxq\" (UID: \"fc7003b6-c170-420a-ba0a-d94cc9b56c36\") " pod="openshift-lws-operator/lws-controller-manager-7475cd49b4-wclxq"
Jul 20 21:11:06.359425 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.359285    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fc7003b6-c170-420a-ba0a-d94cc9b56c36-metrics-cert\") pod \"lws-controller-manager-7475cd49b4-wclxq\" (UID: \"fc7003b6-c170-420a-ba0a-d94cc9b56c36\") " pod="openshift-lws-operator/lws-controller-manager-7475cd49b4-wclxq"
Jul 20 21:11:06.459659 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.459614    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fc7003b6-c170-420a-ba0a-d94cc9b56c36-metrics-cert\") pod \"lws-controller-manager-7475cd49b4-wclxq\" (UID: \"fc7003b6-c170-420a-ba0a-d94cc9b56c36\") " pod="openshift-lws-operator/lws-controller-manager-7475cd49b4-wclxq"
Jul 20 21:11:06.459820 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.459675    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fc7003b6-c170-420a-ba0a-d94cc9b56c36-manager-config\") pod \"lws-controller-manager-7475cd49b4-wclxq\" (UID: \"fc7003b6-c170-420a-ba0a-d94cc9b56c36\") " pod="openshift-lws-operator/lws-controller-manager-7475cd49b4-wclxq"
Jul 20 21:11:06.459820 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.459694    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc7003b6-c170-420a-ba0a-d94cc9b56c36-cert\") pod \"lws-controller-manager-7475cd49b4-wclxq\" (UID: \"fc7003b6-c170-420a-ba0a-d94cc9b56c36\") " pod="openshift-lws-operator/lws-controller-manager-7475cd49b4-wclxq"
Jul 20 21:11:06.459820 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.459729    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9shp\" (UniqueName: \"kubernetes.io/projected/fc7003b6-c170-420a-ba0a-d94cc9b56c36-kube-api-access-f9shp\") pod \"lws-controller-manager-7475cd49b4-wclxq\" (UID: \"fc7003b6-c170-420a-ba0a-d94cc9b56c36\") " pod="openshift-lws-operator/lws-controller-manager-7475cd49b4-wclxq"
Jul 20 21:11:06.460422 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.460399    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fc7003b6-c170-420a-ba0a-d94cc9b56c36-manager-config\") pod \"lws-controller-manager-7475cd49b4-wclxq\" (UID: \"fc7003b6-c170-420a-ba0a-d94cc9b56c36\") " pod="openshift-lws-operator/lws-controller-manager-7475cd49b4-wclxq"
Jul 20 21:11:06.462117 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.462095    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fc7003b6-c170-420a-ba0a-d94cc9b56c36-metrics-cert\") pod \"lws-controller-manager-7475cd49b4-wclxq\" (UID: \"fc7003b6-c170-420a-ba0a-d94cc9b56c36\") " pod="openshift-lws-operator/lws-controller-manager-7475cd49b4-wclxq"
Jul 20 21:11:06.462206 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.462188    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc7003b6-c170-420a-ba0a-d94cc9b56c36-cert\") pod \"lws-controller-manager-7475cd49b4-wclxq\" (UID: \"fc7003b6-c170-420a-ba0a-d94cc9b56c36\") " pod="openshift-lws-operator/lws-controller-manager-7475cd49b4-wclxq"
Jul 20 21:11:06.471905 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.471879    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9shp\" (UniqueName: \"kubernetes.io/projected/fc7003b6-c170-420a-ba0a-d94cc9b56c36-kube-api-access-f9shp\") pod \"lws-controller-manager-7475cd49b4-wclxq\" (UID: \"fc7003b6-c170-420a-ba0a-d94cc9b56c36\") " pod="openshift-lws-operator/lws-controller-manager-7475cd49b4-wclxq"
Jul 20 21:11:06.578377 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.578350    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7475cd49b4-wclxq"
Jul 20 21:11:06.720487 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:06.720452    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7475cd49b4-wclxq"]
Jul 20 21:11:06.723356 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:11:06.723331    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc7003b6_c170_420a_ba0a_d94cc9b56c36.slice/crio-7dbc838a94291f320c8f24db149304b8cfb5dbc8c27946dac9cd6bb3a4a50b04 WatchSource:0}: Error finding container 7dbc838a94291f320c8f24db149304b8cfb5dbc8c27946dac9cd6bb3a4a50b04: Status 404 returned error can't find the container with id 7dbc838a94291f320c8f24db149304b8cfb5dbc8c27946dac9cd6bb3a4a50b04
Jul 20 21:11:07.348776 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.348462    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8"]
Jul 20 21:11:07.352458 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.352427    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8"
Jul 20 21:11:07.357426 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.357399    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 20 21:11:07.357705 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.357683    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-tc9ds\""
Jul 20 21:11:07.357980 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.357960    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 20 21:11:07.358059 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.357996    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 20 21:11:07.362181 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.361903    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 20 21:11:07.381615 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.381591    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8"]
Jul 20 21:11:07.466796 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.466756    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8c72\" (UniqueName: \"kubernetes.io/projected/e7b15cf5-17da-4dc2-ab69-25169d4a70bb-kube-api-access-w8c72\") pod \"opendatahub-operator-controller-manager-66b5c4b596-jc2h8\" (UID: \"e7b15cf5-17da-4dc2-ab69-25169d4a70bb\") " pod="opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8"
Jul 20 21:11:07.466972 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.466837    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e7b15cf5-17da-4dc2-ab69-25169d4a70bb-manifests\") pod \"opendatahub-operator-controller-manager-66b5c4b596-jc2h8\" (UID: \"e7b15cf5-17da-4dc2-ab69-25169d4a70bb\") " pod="opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8"
Jul 20 21:11:07.466972 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.466900    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7b15cf5-17da-4dc2-ab69-25169d4a70bb-tmp\") pod \"opendatahub-operator-controller-manager-66b5c4b596-jc2h8\" (UID: \"e7b15cf5-17da-4dc2-ab69-25169d4a70bb\") " pod="opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8"
Jul 20 21:11:07.466972 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.466933    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7b15cf5-17da-4dc2-ab69-25169d4a70bb-webhook-cert\") pod \"opendatahub-operator-controller-manager-66b5c4b596-jc2h8\" (UID: \"e7b15cf5-17da-4dc2-ab69-25169d4a70bb\") " pod="opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8"
Jul 20 21:11:07.466972 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.466966    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e7b15cf5-17da-4dc2-ab69-25169d4a70bb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66b5c4b596-jc2h8\" (UID: \"e7b15cf5-17da-4dc2-ab69-25169d4a70bb\") " pod="opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8"
Jul 20 21:11:07.568313 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.568274    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8c72\" (UniqueName: \"kubernetes.io/projected/e7b15cf5-17da-4dc2-ab69-25169d4a70bb-kube-api-access-w8c72\") pod \"opendatahub-operator-controller-manager-66b5c4b596-jc2h8\" (UID: \"e7b15cf5-17da-4dc2-ab69-25169d4a70bb\") " pod="opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8"
Jul 20 21:11:07.568484 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.568341    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e7b15cf5-17da-4dc2-ab69-25169d4a70bb-manifests\") pod \"opendatahub-operator-controller-manager-66b5c4b596-jc2h8\" (UID: \"e7b15cf5-17da-4dc2-ab69-25169d4a70bb\") " pod="opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8"
Jul 20 21:11:07.568484 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.568393    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7b15cf5-17da-4dc2-ab69-25169d4a70bb-tmp\") pod \"opendatahub-operator-controller-manager-66b5c4b596-jc2h8\" (UID: \"e7b15cf5-17da-4dc2-ab69-25169d4a70bb\") " pod="opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8"
Jul 20 21:11:07.568484 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.568425    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7b15cf5-17da-4dc2-ab69-25169d4a70bb-webhook-cert\") pod \"opendatahub-operator-controller-manager-66b5c4b596-jc2h8\" (UID: \"e7b15cf5-17da-4dc2-ab69-25169d4a70bb\") " pod="opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8"
Jul 20 21:11:07.568484 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.568451    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e7b15cf5-17da-4dc2-ab69-25169d4a70bb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66b5c4b596-jc2h8\" (UID: \"e7b15cf5-17da-4dc2-ab69-25169d4a70bb\") " pod="opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8"
Jul 20 21:11:07.568820 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.568746    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e7b15cf5-17da-4dc2-ab69-25169d4a70bb-manifests\") pod \"opendatahub-operator-controller-manager-66b5c4b596-jc2h8\" (UID: \"e7b15cf5-17da-4dc2-ab69-25169d4a70bb\") " pod="opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8"
Jul 20 21:11:07.571668 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.571610    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e7b15cf5-17da-4dc2-ab69-25169d4a70bb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66b5c4b596-jc2h8\" (UID: \"e7b15cf5-17da-4dc2-ab69-25169d4a70bb\") " pod="opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8"
Jul 20 21:11:07.572305 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.572281    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7b15cf5-17da-4dc2-ab69-25169d4a70bb-tmp\") pod \"opendatahub-operator-controller-manager-66b5c4b596-jc2h8\" (UID: \"e7b15cf5-17da-4dc2-ab69-25169d4a70bb\") " pod="opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8"
Jul 20 21:11:07.574551 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.574503    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7b15cf5-17da-4dc2-ab69-25169d4a70bb-webhook-cert\") pod \"opendatahub-operator-controller-manager-66b5c4b596-jc2h8\" (UID: \"e7b15cf5-17da-4dc2-ab69-25169d4a70bb\") " pod="opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8"
Jul 20 21:11:07.584827 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.584771    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8c72\" (UniqueName: \"kubernetes.io/projected/e7b15cf5-17da-4dc2-ab69-25169d4a70bb-kube-api-access-w8c72\") pod \"opendatahub-operator-controller-manager-66b5c4b596-jc2h8\" (UID: \"e7b15cf5-17da-4dc2-ab69-25169d4a70bb\") " pod="opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8"
Jul 20 21:11:07.662578 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.662486    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7475cd49b4-wclxq" event={"ID":"fc7003b6-c170-420a-ba0a-d94cc9b56c36","Type":"ContainerStarted","Data":"7dbc838a94291f320c8f24db149304b8cfb5dbc8c27946dac9cd6bb3a4a50b04"}
Jul 20 21:11:07.666758 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.666728    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8"
Jul 20 21:11:07.856985 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:07.856945    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8"]
Jul 20 21:11:07.862461 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:11:07.862427    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7b15cf5_17da_4dc2_ab69_25169d4a70bb.slice/crio-53fa91b3e405706439e6c89cf47fd0b1e3c9c82c8a0b9821d97228a79464bc9e WatchSource:0}: Error finding container 53fa91b3e405706439e6c89cf47fd0b1e3c9c82c8a0b9821d97228a79464bc9e: Status 404 returned error can't find the container with id 53fa91b3e405706439e6c89cf47fd0b1e3c9c82c8a0b9821d97228a79464bc9e
Jul 20 21:11:08.669433 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:08.669391    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8" event={"ID":"e7b15cf5-17da-4dc2-ab69-25169d4a70bb","Type":"ContainerStarted","Data":"53fa91b3e405706439e6c89cf47fd0b1e3c9c82c8a0b9821d97228a79464bc9e"}
Jul 20 21:11:09.678181 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:09.678133    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7475cd49b4-wclxq" event={"ID":"fc7003b6-c170-420a-ba0a-d94cc9b56c36","Type":"ContainerStarted","Data":"f83d1f9ac9b4c2b1ff36bb41bd6e34cb7335abc0fe192b07252a112ccdd91ada"}
Jul 20 21:11:09.678621 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:09.678431    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7475cd49b4-wclxq"
Jul 20 21:11:09.711276 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:09.711223    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7475cd49b4-wclxq" podStartSLOduration=1.428270105 podStartE2EDuration="3.711206039s" totalImagesPullingTime="2.282935934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:11:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:11:09.702014827 +0000 UTC m=+510.606004408" watchObservedRunningTime="2026-07-20 21:11:09.711206039 +0000 UTC m=+510.615195620"
Jul 20 21:11:11.689662 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:11.689578    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8" event={"ID":"e7b15cf5-17da-4dc2-ab69-25169d4a70bb","Type":"ContainerStarted","Data":"da72d192a64f5015f612e49c03e5c9f598437b0fed65da3f5139d754604d446f"}
Jul 20 21:11:12.695657 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:12.695624    2561 generic.go:356] "Generic (PLEG): container finished" podID="e7b15cf5-17da-4dc2-ab69-25169d4a70bb" containerID="da72d192a64f5015f612e49c03e5c9f598437b0fed65da3f5139d754604d446f" exitCode=0
Jul 20 21:11:12.696052 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:12.695707    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8" event={"ID":"e7b15cf5-17da-4dc2-ab69-25169d4a70bb","Type":"ContainerDied","Data":"da72d192a64f5015f612e49c03e5c9f598437b0fed65da3f5139d754604d446f"}
Jul 20 21:11:13.702626 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:13.702591    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8" event={"ID":"e7b15cf5-17da-4dc2-ab69-25169d4a70bb","Type":"ContainerStarted","Data":"8d20a5e68f320f92e38ab40858be60bdd94f4fe6c7b29df32d9104a86bd7e5ab"}
Jul 20 21:11:13.703051 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:13.702688    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8"
Jul 20 21:11:13.731337 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:13.731284    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8" podStartSLOduration=1.709930259 podStartE2EDuration="6.731270286s" totalImagesPullingTime="3.961818972s" totalInitContainerRuntime="1.059521055s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:11:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:11:13.728511551 +0000 UTC m=+514.632501152" watchObservedRunningTime="2026-07-20 21:11:13.731270286 +0000 UTC m=+514.635259863"
Jul 20 21:11:20.686834 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:20.686805    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7475cd49b4-wclxq"
Jul 20 21:11:24.708505 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:24.708474    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-66b5c4b596-jc2h8"
Jul 20 21:11:36.594957 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.594881    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"]
Jul 20 21:11:36.598124 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.598105    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.600658 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.600629    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 20 21:11:36.600658 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.600647    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-72gfw\""
Jul 20 21:11:36.612366 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.612338    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"]
Jul 20 21:11:36.723624 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.723589    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.723778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.723635    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.723778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.723669    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.723778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.723715    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.723778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.723740    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.723986 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.723792    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.723986 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.723820    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.723986 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.723895    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxl45\" (UniqueName: \"kubernetes.io/projected/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-kube-api-access-mxl45\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.724169 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.723991    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.816125 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.816099    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"]
Jul 20 21:11:36.819350 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.819334    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"
Jul 20 21:11:36.822606 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.822587    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 20 21:11:36.822714 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.822657    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-r57pv\""
Jul 20 21:11:36.823033 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.823016    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 20 21:11:36.823309 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.823286    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 20 21:11:36.825324 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.825305    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxl45\" (UniqueName: \"kubernetes.io/projected/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-kube-api-access-mxl45\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.825401 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.825355    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.825447 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.825403    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.825447 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.825433    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.825551 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.825465    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.825551 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.825500    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.825551 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.825531    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.825707 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.825561    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.825707 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.825589    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.826032 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.826006    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.826139 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.826038    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.826386 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.826365    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.826455 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.826418    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.826455 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.826433    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.828313 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.828294    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.828584 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.828569    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.840178 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.840155    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"]
Jul 20 21:11:36.843576 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.843545    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.844026 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.844004    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxl45\" (UniqueName: \"kubernetes.io/projected/f0fb0c79-9936-407c-a54e-ee4dc5cabc03-kube-api-access-mxl45\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4c9892\" (UID: \"f0fb0c79-9936-407c-a54e-ee4dc5cabc03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.910251 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.910184    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:36.926372 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.926340    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtkbd\" (UniqueName: \"kubernetes.io/projected/70b96039-dacf-4054-8ffb-e9f20695be71-kube-api-access-rtkbd\") pod \"istiod-openshift-gateway-94bb8fbfd-jp58g\" (UID: \"70b96039-dacf-4054-8ffb-e9f20695be71\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"
Jul 20 21:11:36.926484 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.926380    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/70b96039-dacf-4054-8ffb-e9f20695be71-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-jp58g\" (UID: \"70b96039-dacf-4054-8ffb-e9f20695be71\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"
Jul 20 21:11:36.926484 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.926409    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/70b96039-dacf-4054-8ffb-e9f20695be71-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-jp58g\" (UID: \"70b96039-dacf-4054-8ffb-e9f20695be71\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"
Jul 20 21:11:36.926575 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.926511    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/70b96039-dacf-4054-8ffb-e9f20695be71-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-jp58g\" (UID: \"70b96039-dacf-4054-8ffb-e9f20695be71\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"
Jul 20 21:11:36.926575 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.926566    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/70b96039-dacf-4054-8ffb-e9f20695be71-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-jp58g\" (UID: \"70b96039-dacf-4054-8ffb-e9f20695be71\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"
Jul 20 21:11:36.926702 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.926602    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/70b96039-dacf-4054-8ffb-e9f20695be71-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-jp58g\" (UID: \"70b96039-dacf-4054-8ffb-e9f20695be71\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"
Jul 20 21:11:36.926702 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:36.926653    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70b96039-dacf-4054-8ffb-e9f20695be71-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-jp58g\" (UID: \"70b96039-dacf-4054-8ffb-e9f20695be71\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"
Jul 20 21:11:37.027871 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:37.027840    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtkbd\" (UniqueName: \"kubernetes.io/projected/70b96039-dacf-4054-8ffb-e9f20695be71-kube-api-access-rtkbd\") pod \"istiod-openshift-gateway-94bb8fbfd-jp58g\" (UID: \"70b96039-dacf-4054-8ffb-e9f20695be71\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"
Jul 20 21:11:37.028026 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:37.027882    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/70b96039-dacf-4054-8ffb-e9f20695be71-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-jp58g\" (UID: \"70b96039-dacf-4054-8ffb-e9f20695be71\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"
Jul 20 21:11:37.028145 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:37.028013    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/70b96039-dacf-4054-8ffb-e9f20695be71-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-jp58g\" (UID: \"70b96039-dacf-4054-8ffb-e9f20695be71\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"
Jul 20 21:11:37.028145 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:37.028116    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/70b96039-dacf-4054-8ffb-e9f20695be71-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-jp58g\" (UID: \"70b96039-dacf-4054-8ffb-e9f20695be71\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"
Jul 20 21:11:37.028254 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:37.028145    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/70b96039-dacf-4054-8ffb-e9f20695be71-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-jp58g\" (UID: \"70b96039-dacf-4054-8ffb-e9f20695be71\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"
Jul 20 21:11:37.028254 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:37.028175    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/70b96039-dacf-4054-8ffb-e9f20695be71-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-jp58g\" (UID: \"70b96039-dacf-4054-8ffb-e9f20695be71\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"
Jul 20 21:11:37.028254 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:37.028230    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70b96039-dacf-4054-8ffb-e9f20695be71-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-jp58g\" (UID: \"70b96039-dacf-4054-8ffb-e9f20695be71\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"
Jul 20 21:11:37.028906 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:37.028881    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/70b96039-dacf-4054-8ffb-e9f20695be71-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-jp58g\" (UID: \"70b96039-dacf-4054-8ffb-e9f20695be71\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"
Jul 20 21:11:37.030396 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:37.030374    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/70b96039-dacf-4054-8ffb-e9f20695be71-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-jp58g\" (UID: \"70b96039-dacf-4054-8ffb-e9f20695be71\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"
Jul 20 21:11:37.030498 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:37.030373    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/70b96039-dacf-4054-8ffb-e9f20695be71-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-jp58g\" (UID: \"70b96039-dacf-4054-8ffb-e9f20695be71\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"
Jul 20 21:11:37.030498 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:37.030379    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/70b96039-dacf-4054-8ffb-e9f20695be71-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-jp58g\" (UID: \"70b96039-dacf-4054-8ffb-e9f20695be71\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"
Jul 20 21:11:37.030498 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:37.030492    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/70b96039-dacf-4054-8ffb-e9f20695be71-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-jp58g\" (UID: \"70b96039-dacf-4054-8ffb-e9f20695be71\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"
Jul 20 21:11:37.046192 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:37.046173    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70b96039-dacf-4054-8ffb-e9f20695be71-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-jp58g\" (UID: \"70b96039-dacf-4054-8ffb-e9f20695be71\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"
Jul 20 21:11:37.046512 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:37.046492    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtkbd\" (UniqueName: \"kubernetes.io/projected/70b96039-dacf-4054-8ffb-e9f20695be71-kube-api-access-rtkbd\") pod \"istiod-openshift-gateway-94bb8fbfd-jp58g\" (UID: \"70b96039-dacf-4054-8ffb-e9f20695be71\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"
Jul 20 21:11:37.057034 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:37.056987    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"]
Jul 20 21:11:37.059231 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:11:37.059208    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0fb0c79_9936_407c_a54e_ee4dc5cabc03.slice/crio-cce54e97e26a8c831031d0b88f22861c9f76c118dffd78c964f1505cc39cd80c WatchSource:0}: Error finding container cce54e97e26a8c831031d0b88f22861c9f76c118dffd78c964f1505cc39cd80c: Status 404 returned error can't find the container with id cce54e97e26a8c831031d0b88f22861c9f76c118dffd78c964f1505cc39cd80c
Jul 20 21:11:37.144713 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:37.144686    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"
Jul 20 21:11:37.278695 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:37.278664    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"]
Jul 20 21:11:37.279430 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:11:37.279408    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70b96039_dacf_4054_8ffb_e9f20695be71.slice/crio-419a93c4d5960cda086a0f184c4293bd15a9fbf5cad5af133e1a1fc667f2d871 WatchSource:0}: Error finding container 419a93c4d5960cda086a0f184c4293bd15a9fbf5cad5af133e1a1fc667f2d871: Status 404 returned error can't find the container with id 419a93c4d5960cda086a0f184c4293bd15a9fbf5cad5af133e1a1fc667f2d871
Jul 20 21:11:37.808533 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:37.808484    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g" event={"ID":"70b96039-dacf-4054-8ffb-e9f20695be71","Type":"ContainerStarted","Data":"419a93c4d5960cda086a0f184c4293bd15a9fbf5cad5af133e1a1fc667f2d871"}
Jul 20 21:11:37.810163 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:37.809998    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892" event={"ID":"f0fb0c79-9936-407c-a54e-ee4dc5cabc03","Type":"ContainerStarted","Data":"cce54e97e26a8c831031d0b88f22861c9f76c118dffd78c964f1505cc39cd80c"}
Jul 20 21:11:40.959101 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:40.957714    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:11:40.959101 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:40.957883    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:11:40.959101 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:40.957980    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:11:40.962426 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:40.959999    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:11:40.962426 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:40.960134    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:11:41.835264 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:41.835223    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892" event={"ID":"f0fb0c79-9936-407c-a54e-ee4dc5cabc03","Type":"ContainerStarted","Data":"af790558e6845ca2937d1fc3e35b33eb121b5c1a71c4a62b330b54ac46bff9b5"}
Jul 20 21:11:41.837067 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:41.837038    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g" event={"ID":"70b96039-dacf-4054-8ffb-e9f20695be71","Type":"ContainerStarted","Data":"0fc93289e64f9856ee9d5807def50bb90305b7cfbdb1673f91539eba8ae2796e"}
Jul 20 21:11:41.837213 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:41.837128    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"
Jul 20 21:11:41.866817 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:41.866764    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892" podStartSLOduration=1.970575835 podStartE2EDuration="5.866746759s" totalImagesPullingTime="3.896170924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:11:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:11:41.865856325 +0000 UTC m=+542.769845903" watchObservedRunningTime="2026-07-20 21:11:41.866746759 +0000 UTC m=+542.770736337"
Jul 20 21:11:41.907613 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:41.907560    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g" podStartSLOduration=2.229261054 podStartE2EDuration="5.907540249s" totalImagesPullingTime="3.678279195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:11:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:11:41.906996627 +0000 UTC m=+542.810986215" watchObservedRunningTime="2026-07-20 21:11:41.907540249 +0000 UTC m=+542.811529827"
Jul 20 21:11:41.911055 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:41.911031    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:42.843652 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:42.843623    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jp58g"
Jul 20 21:11:42.915759 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:42.915727    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:43.846606 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:43.846570    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:11:43.847732 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:11:43.847709    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4c9892"
Jul 20 21:12:40.331776 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:12:40.331662    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/1.log"
Jul 20 21:12:40.333831 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:12:40.332335    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/1.log"
Jul 20 21:13:04.715709 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:04.715676    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bfbc5df8c-gsnwf"]
Jul 20 21:13:29.735388 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:29.735329    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6bfbc5df8c-gsnwf" podUID="e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d" containerName="console" containerID="cri-o://412a47780e90f7b9462d0dbf44a1a44342a5052571a8b805e6d99f0d31b808b2" gracePeriod=15
Jul 20 21:13:29.986394 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:29.986337    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:13:30.127759 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.127724    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-console-serving-cert\") pod \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") "
Jul 20 21:13:30.127927 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.127772    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-service-ca\") pod \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") "
Jul 20 21:13:30.127927 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.127790    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-oauth-serving-cert\") pod \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") "
Jul 20 21:13:30.127927 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.127857    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-console-oauth-config\") pod \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") "
Jul 20 21:13:30.127927 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.127877    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfwr7\" (UniqueName: \"kubernetes.io/projected/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-kube-api-access-dfwr7\") pod \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") "
Jul 20 21:13:30.128142 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.127928    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-console-config\") pod \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") "
Jul 20 21:13:30.128142 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.127950    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-trusted-ca-bundle\") pod \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\" (UID: \"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d\") "
Jul 20 21:13:30.128424 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.128303    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-service-ca" (OuterVolumeSpecName: "service-ca") pod "e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d" (UID: "e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 21:13:30.128424 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.128390    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-console-config" (OuterVolumeSpecName: "console-config") pod "e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d" (UID: "e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 21:13:30.128561 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.128312    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d" (UID: "e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 21:13:30.128561 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.128489    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d" (UID: "e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 21:13:30.129859 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.129834    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d" (UID: "e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 21:13:30.130415 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.130385    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d" (UID: "e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 21:13:30.130499 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.130396    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-kube-api-access-dfwr7" (OuterVolumeSpecName: "kube-api-access-dfwr7") pod "e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d" (UID: "e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d"). InnerVolumeSpecName "kube-api-access-dfwr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 21:13:30.228578 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.228550    2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-service-ca\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:13:30.228578 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.228575    2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-oauth-serving-cert\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:13:30.228578 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.228584    2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-console-oauth-config\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:13:30.228758 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.228593    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfwr7\" (UniqueName: \"kubernetes.io/projected/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-kube-api-access-dfwr7\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:13:30.228758 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.228602    2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-console-config\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:13:30.228758 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.228610    2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-trusted-ca-bundle\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:13:30.228758 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.228617    2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d-console-serving-cert\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:13:30.340546 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.340351    2561 generic.go:356] "Generic (PLEG): container finished" podID="e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d" containerID="412a47780e90f7b9462d0dbf44a1a44342a5052571a8b805e6d99f0d31b808b2" exitCode=0
Jul 20 21:13:30.340744 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.340555    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bfbc5df8c-gsnwf" event={"ID":"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d","Type":"ContainerDied","Data":"412a47780e90f7b9462d0dbf44a1a44342a5052571a8b805e6d99f0d31b808b2"}
Jul 20 21:13:30.340808 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.340752    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bfbc5df8c-gsnwf" event={"ID":"e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d","Type":"ContainerDied","Data":"14076af6c8e367b4c5a3ac3a81b65af0bd68b4c22357814291deb903070ec538"}
Jul 20 21:13:30.340808 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.340756    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bfbc5df8c-gsnwf"
Jul 20 21:13:30.340808 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.340794    2561 scope.go:117] "RemoveContainer" containerID="412a47780e90f7b9462d0dbf44a1a44342a5052571a8b805e6d99f0d31b808b2"
Jul 20 21:13:30.356212 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.356189    2561 scope.go:117] "RemoveContainer" containerID="412a47780e90f7b9462d0dbf44a1a44342a5052571a8b805e6d99f0d31b808b2"
Jul 20 21:13:30.356521 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:13:30.356502    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"412a47780e90f7b9462d0dbf44a1a44342a5052571a8b805e6d99f0d31b808b2\": container with ID starting with 412a47780e90f7b9462d0dbf44a1a44342a5052571a8b805e6d99f0d31b808b2 not found: ID does not exist" containerID="412a47780e90f7b9462d0dbf44a1a44342a5052571a8b805e6d99f0d31b808b2"
Jul 20 21:13:30.356580 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.356529    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"412a47780e90f7b9462d0dbf44a1a44342a5052571a8b805e6d99f0d31b808b2"} err="failed to get container status \"412a47780e90f7b9462d0dbf44a1a44342a5052571a8b805e6d99f0d31b808b2\": rpc error: code = NotFound desc = could not find container \"412a47780e90f7b9462d0dbf44a1a44342a5052571a8b805e6d99f0d31b808b2\": container with ID starting with 412a47780e90f7b9462d0dbf44a1a44342a5052571a8b805e6d99f0d31b808b2 not found: ID does not exist"
Jul 20 21:13:30.366470 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.366442    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bfbc5df8c-gsnwf"]
Jul 20 21:13:30.371825 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:30.371803    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6bfbc5df8c-gsnwf"]
Jul 20 21:13:31.726139 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:31.726106    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d" path="/var/lib/kubelet/pods/e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d/volumes"
Jul 20 21:13:39.558051 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:13:39.558007    2561 scope.go:117] "RemoveContainer" containerID="1bd226643cae56a96bd43b76d48155de9b1598642f9916e28ab913f4362e02a2"
Jul 20 21:14:30.113470 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:30.113433    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:14:30.113892 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:30.113757    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d" containerName="console"
Jul 20 21:14:30.113892 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:30.113768    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d" containerName="console"
Jul 20 21:14:30.113978 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:30.113894    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e78c13b1-73aa-4cbb-9bd4-2dadf0e4970d" containerName="console"
Jul 20 21:14:30.117051 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:30.117033    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"
Jul 20 21:14:30.119732 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:30.119710    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 20 21:14:30.119851 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:30.119741    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 20 21:14:30.119851 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:30.119710    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 20 21:14:30.120568 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:30.120545    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7cj5x\""
Jul 20 21:14:30.127192 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:30.127171    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:14:30.146667 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:30.146642    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prg65\" (UniqueName: \"kubernetes.io/projected/4b486af1-6a3f-41c8-8ba9-b428fd7abfcb-kube-api-access-prg65\") pod \"limitador-limitador-54d6d988d5-9cb2q\" (UID: \"4b486af1-6a3f-41c8-8ba9-b428fd7abfcb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"
Jul 20 21:14:30.146759 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:30.146681    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4b486af1-6a3f-41c8-8ba9-b428fd7abfcb-config-file\") pod \"limitador-limitador-54d6d988d5-9cb2q\" (UID: \"4b486af1-6a3f-41c8-8ba9-b428fd7abfcb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"
Jul 20 21:14:30.165017 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:30.164992    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:14:30.247377 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:30.247346    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prg65\" (UniqueName: \"kubernetes.io/projected/4b486af1-6a3f-41c8-8ba9-b428fd7abfcb-kube-api-access-prg65\") pod \"limitador-limitador-54d6d988d5-9cb2q\" (UID: \"4b486af1-6a3f-41c8-8ba9-b428fd7abfcb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"
Jul 20 21:14:30.247536 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:30.247422    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4b486af1-6a3f-41c8-8ba9-b428fd7abfcb-config-file\") pod \"limitador-limitador-54d6d988d5-9cb2q\" (UID: \"4b486af1-6a3f-41c8-8ba9-b428fd7abfcb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"
Jul 20 21:14:30.248059 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:30.248035    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4b486af1-6a3f-41c8-8ba9-b428fd7abfcb-config-file\") pod \"limitador-limitador-54d6d988d5-9cb2q\" (UID: \"4b486af1-6a3f-41c8-8ba9-b428fd7abfcb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"
Jul 20 21:14:30.256316 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:30.256295    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prg65\" (UniqueName: \"kubernetes.io/projected/4b486af1-6a3f-41c8-8ba9-b428fd7abfcb-kube-api-access-prg65\") pod \"limitador-limitador-54d6d988d5-9cb2q\" (UID: \"4b486af1-6a3f-41c8-8ba9-b428fd7abfcb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"
Jul 20 21:14:30.427091 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:30.426982    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"
Jul 20 21:14:30.556466 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:30.556349    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:14:30.559280 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:14:30.559250    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b486af1_6a3f_41c8_8ba9_b428fd7abfcb.slice/crio-66ea30d15a0c60f732f232154f4c67338de7b3212fbe8257ba679b5eb04afa44 WatchSource:0}: Error finding container 66ea30d15a0c60f732f232154f4c67338de7b3212fbe8257ba679b5eb04afa44: Status 404 returned error can't find the container with id 66ea30d15a0c60f732f232154f4c67338de7b3212fbe8257ba679b5eb04afa44
Jul 20 21:14:30.561414 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:30.561395    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 21:14:30.618376 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:30.618338    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-9cb2q" event={"ID":"4b486af1-6a3f-41c8-8ba9-b428fd7abfcb","Type":"ContainerStarted","Data":"66ea30d15a0c60f732f232154f4c67338de7b3212fbe8257ba679b5eb04afa44"}
Jul 20 21:14:33.635462 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:33.635425    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-9cb2q" event={"ID":"4b486af1-6a3f-41c8-8ba9-b428fd7abfcb","Type":"ContainerStarted","Data":"54aa32d451ab498b51cc2f19e93e966c5238435d102e8d4fe7f4adbc618f37fa"}
Jul 20 21:14:33.635876 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:33.635511    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"
Jul 20 21:14:33.655462 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:33.655413    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-9cb2q" podStartSLOduration=1.100028763 podStartE2EDuration="3.655398677s" totalImagesPullingTime="2.555369914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:14:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:14:33.654260139 +0000 UTC m=+714.558249716" watchObservedRunningTime="2026-07-20 21:14:33.655398677 +0000 UTC m=+714.559388255"
Jul 20 21:14:39.578133 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:39.578104    2561 scope.go:117] "RemoveContainer" containerID="fc9a4f0c121969ff4998062ab1c8441ec4f8f31ab74bd1c8286b97b7ea85c8cb"
Jul 20 21:14:39.586040 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:39.586019    2561 scope.go:117] "RemoveContainer" containerID="fd0ecd5b74553559d132ae51314c0ac1d5103c822c9d8452754627a68bbc16de"
Jul 20 21:14:39.592886 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:39.592869    2561 scope.go:117] "RemoveContainer" containerID="90ee6d516f88707567c0178ec83439559d7286bd8cf76650b27e61f57bcf883a"
Jul 20 21:14:39.599288 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:39.599272    2561 scope.go:117] "RemoveContainer" containerID="49a530a34e0db9220b6e676b3c3a96e9cc436094e9fdfc7f762eabb4c65d545f"
Jul 20 21:14:39.605759 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:39.605743    2561 scope.go:117] "RemoveContainer" containerID="54269cdbf4daeb678b535006f744a2caae3eac341b21207e6bb0ea5dedd61190"
Jul 20 21:14:39.612638 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:39.612617    2561 scope.go:117] "RemoveContainer" containerID="8064381ce9898b95ad571daf250376034c2eebe0d207401dde19a7119dc208c8"
Jul 20 21:14:44.641422 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:14:44.641391    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"
Jul 20 21:15:46.280693 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:15:46.280655    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:16:11.193986 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:11.193949    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:16:14.892324 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:14.892277    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:16:22.584473 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:22.584432    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:16:30.793777 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:30.793735    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:16:46.683019 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.682988    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"]
Jul 20 21:16:46.686399 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.686383    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"
Jul 20 21:16:46.688845 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.688826    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-fmhkv\""
Jul 20 21:16:46.688961 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.688875    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 20 21:16:46.689816 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.689798    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 20 21:16:46.689918 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.689879    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 20 21:16:46.695337 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.695313    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"]
Jul 20 21:16:46.799626 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.799584    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7dbbd3a4-752c-4a19-bf00-5fa435dc4258-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lfktv\" (UID: \"7dbbd3a4-752c-4a19-bf00-5fa435dc4258\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"
Jul 20 21:16:46.799626 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.799626    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7dbbd3a4-752c-4a19-bf00-5fa435dc4258-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lfktv\" (UID: \"7dbbd3a4-752c-4a19-bf00-5fa435dc4258\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"
Jul 20 21:16:46.799866 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.799662    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfrxb\" (UniqueName: \"kubernetes.io/projected/7dbbd3a4-752c-4a19-bf00-5fa435dc4258-kube-api-access-jfrxb\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lfktv\" (UID: \"7dbbd3a4-752c-4a19-bf00-5fa435dc4258\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"
Jul 20 21:16:46.799866 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.799716    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7dbbd3a4-752c-4a19-bf00-5fa435dc4258-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lfktv\" (UID: \"7dbbd3a4-752c-4a19-bf00-5fa435dc4258\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"
Jul 20 21:16:46.799866 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.799766    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dbbd3a4-752c-4a19-bf00-5fa435dc4258-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lfktv\" (UID: \"7dbbd3a4-752c-4a19-bf00-5fa435dc4258\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"
Jul 20 21:16:46.799866 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.799837    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7dbbd3a4-752c-4a19-bf00-5fa435dc4258-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lfktv\" (UID: \"7dbbd3a4-752c-4a19-bf00-5fa435dc4258\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"
Jul 20 21:16:46.900331 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.900293    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7dbbd3a4-752c-4a19-bf00-5fa435dc4258-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lfktv\" (UID: \"7dbbd3a4-752c-4a19-bf00-5fa435dc4258\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"
Jul 20 21:16:46.900331 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.900334    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfrxb\" (UniqueName: \"kubernetes.io/projected/7dbbd3a4-752c-4a19-bf00-5fa435dc4258-kube-api-access-jfrxb\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lfktv\" (UID: \"7dbbd3a4-752c-4a19-bf00-5fa435dc4258\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"
Jul 20 21:16:46.900626 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.900374    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7dbbd3a4-752c-4a19-bf00-5fa435dc4258-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lfktv\" (UID: \"7dbbd3a4-752c-4a19-bf00-5fa435dc4258\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"
Jul 20 21:16:46.900626 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.900429    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dbbd3a4-752c-4a19-bf00-5fa435dc4258-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lfktv\" (UID: \"7dbbd3a4-752c-4a19-bf00-5fa435dc4258\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"
Jul 20 21:16:46.900626 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.900495    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7dbbd3a4-752c-4a19-bf00-5fa435dc4258-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lfktv\" (UID: \"7dbbd3a4-752c-4a19-bf00-5fa435dc4258\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"
Jul 20 21:16:46.900626 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.900536    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7dbbd3a4-752c-4a19-bf00-5fa435dc4258-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lfktv\" (UID: \"7dbbd3a4-752c-4a19-bf00-5fa435dc4258\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"
Jul 20 21:16:46.900875 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.900853    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dbbd3a4-752c-4a19-bf00-5fa435dc4258-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lfktv\" (UID: \"7dbbd3a4-752c-4a19-bf00-5fa435dc4258\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"
Jul 20 21:16:46.901018 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.901000    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7dbbd3a4-752c-4a19-bf00-5fa435dc4258-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lfktv\" (UID: \"7dbbd3a4-752c-4a19-bf00-5fa435dc4258\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"
Jul 20 21:16:46.901098 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.901002    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7dbbd3a4-752c-4a19-bf00-5fa435dc4258-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lfktv\" (UID: \"7dbbd3a4-752c-4a19-bf00-5fa435dc4258\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"
Jul 20 21:16:46.902780 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.902754    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7dbbd3a4-752c-4a19-bf00-5fa435dc4258-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lfktv\" (UID: \"7dbbd3a4-752c-4a19-bf00-5fa435dc4258\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"
Jul 20 21:16:46.902940 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.902925    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7dbbd3a4-752c-4a19-bf00-5fa435dc4258-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lfktv\" (UID: \"7dbbd3a4-752c-4a19-bf00-5fa435dc4258\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"
Jul 20 21:16:46.910404 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.910379    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfrxb\" (UniqueName: \"kubernetes.io/projected/7dbbd3a4-752c-4a19-bf00-5fa435dc4258-kube-api-access-jfrxb\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lfktv\" (UID: \"7dbbd3a4-752c-4a19-bf00-5fa435dc4258\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"
Jul 20 21:16:46.999373 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:46.999294    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"
Jul 20 21:16:47.132782 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:47.132757    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"]
Jul 20 21:16:47.135020 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:16:47.134990    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dbbd3a4_752c_4a19_bf00_5fa435dc4258.slice/crio-7d398cd25de38b3cc8e136311de6fdc8684a89c2f9d58d7e22d29286f98fdeb3 WatchSource:0}: Error finding container 7d398cd25de38b3cc8e136311de6fdc8684a89c2f9d58d7e22d29286f98fdeb3: Status 404 returned error can't find the container with id 7d398cd25de38b3cc8e136311de6fdc8684a89c2f9d58d7e22d29286f98fdeb3
Jul 20 21:16:47.221713 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:47.221675    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv" event={"ID":"7dbbd3a4-752c-4a19-bf00-5fa435dc4258","Type":"ContainerStarted","Data":"7d398cd25de38b3cc8e136311de6fdc8684a89c2f9d58d7e22d29286f98fdeb3"}
Jul 20 21:16:49.234268 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:49.234221    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv" event={"ID":"7dbbd3a4-752c-4a19-bf00-5fa435dc4258","Type":"ContainerStarted","Data":"c52d343d0f97168abde6f0e09160b12994591fa144cee74b2721021f993ef359"}
Jul 20 21:16:49.252371 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:49.252322    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv" podStartSLOduration=2.219190678 podStartE2EDuration="3.252307467s" totalImagesPullingTime="1.033116789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:16:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:16:49.250876316 +0000 UTC m=+850.154865917" watchObservedRunningTime="2026-07-20 21:16:49.252307467 +0000 UTC m=+850.156297045"
Jul 20 21:16:49.575582 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:49.575550    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:16:56.999654 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:56.999618    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"
Jul 20 21:16:57.000142 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:56.999676    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"
Jul 20 21:16:57.012315 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:57.012288    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"
Jul 20 21:16:57.282947 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:16:57.282864    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lfktv"
Jul 20 21:17:01.482048 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:17:01.482011    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:17:40.366217 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:17:40.366109    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/1.log"
Jul 20 21:17:40.370310 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:17:40.367907    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/1.log"
Jul 20 21:18:41.650717 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:18:41.650682    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:18:51.708879 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:18:51.708840    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:19:03.408350 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:19:03.408312    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:20:47.603597 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:20:47.603525    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:21:24.187866 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:21:24.187830    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:21:30.886231 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:21:30.886193    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:21:40.193656 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:21:40.193620    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:21:51.480566 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:21:51.480527    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:22:01.385830 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:22:01.385748    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:22:13.262705 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:22:13.262669    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:22:23.896556 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:22:23.896522    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:22:34.184573 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:22:34.184539    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:22:40.393784 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:22:40.393681    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/1.log"
Jul 20 21:22:40.397900 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:22:40.396700    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/1.log"
Jul 20 21:22:56.584533 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:22:56.584497    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:23:05.981662 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:23:05.981632    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:23:29.391211 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:23:29.391172    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:23:40.690139 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:23:40.690046    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:23:56.383626 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:23:56.383592    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:24:08.892228 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:24:08.892191    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:24:23.584997 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:24:23.584963    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:26:25.188549 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:26:25.188514    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:26:36.893486 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:26:36.893403    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:26:48.204284 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:26:48.204249    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:27:01.000223 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:27:01.000186    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:27:13.284351 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:27:13.284314    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:27:23.087667 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:27:23.087630    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:27:32.587598 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:27:32.587564    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:27:40.421094 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:27:40.420975    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/1.log"
Jul 20 21:27:40.424547 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:27:40.424526    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/1.log"
Jul 20 21:27:42.385287 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:27:42.385256    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:27:53.392571 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:27:53.392539    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:28:02.297413 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:28:02.297329    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:28:12.782834 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:28:12.782797    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:28:38.432092 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:28:38.428594    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:28:46.290606 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:28:46.290565    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:29:01.321693 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:01.321656    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:29:06.301382 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:06.301349    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"]
Jul 20 21:29:06.304757 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:06.304739    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:06.308255 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:06.308237    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-690af3af-kserve-self-signed-certs\""
Jul 20 21:29:06.316306 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:06.316282    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"]
Jul 20 21:29:06.418936 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:06.418897    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfhgs\" (UniqueName: \"kubernetes.io/projected/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-kube-api-access-kfhgs\") pod \"e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v\" (UID: \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\") " pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:06.419169 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:06.418958    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-dshm\") pod \"e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v\" (UID: \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\") " pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:06.419169 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:06.418996    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-tls-certs\") pod \"e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v\" (UID: \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\") " pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:06.419169 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:06.419020    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-model-cache\") pod \"e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v\" (UID: \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\") " pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:06.419169 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:06.419143    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-tmp-dir\") pod \"e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v\" (UID: \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\") " pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:06.419355 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:06.419174    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-home\") pod \"e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v\" (UID: \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\") " pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:06.519858 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:06.519824    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-dshm\") pod \"e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v\" (UID: \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\") " pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:06.519858 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:06.519862    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-tls-certs\") pod \"e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v\" (UID: \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\") " pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:06.520147 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:06.519877    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-model-cache\") pod \"e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v\" (UID: \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\") " pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:06.520147 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:06.519912    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-tmp-dir\") pod \"e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v\" (UID: \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\") " pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:06.520147 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:06.519935    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-home\") pod \"e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v\" (UID: \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\") " pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:06.520147 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:06.519972    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfhgs\" (UniqueName: \"kubernetes.io/projected/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-kube-api-access-kfhgs\") pod \"e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v\" (UID: \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\") " pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:06.520412 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:06.520390    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-model-cache\") pod \"e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v\" (UID: \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\") " pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:06.520466 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:06.520414    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-tmp-dir\") pod \"e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v\" (UID: \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\") " pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:06.520510 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:06.520475    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-home\") pod \"e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v\" (UID: \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\") " pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:06.522181 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:06.522158    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-dshm\") pod \"e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v\" (UID: \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\") " pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:06.522463 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:06.522446    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-tls-certs\") pod \"e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v\" (UID: \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\") " pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:06.529665 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:06.529643    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfhgs\" (UniqueName: \"kubernetes.io/projected/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-kube-api-access-kfhgs\") pod \"e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v\" (UID: \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\") " pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:06.614294 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:06.614205    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:06.762794 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:06.762761    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"]
Jul 20 21:29:06.764946 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:29:06.764917    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod64e6a540_81a0_4b44_b6b0_79b6e7c4a849.slice/crio-af15560b9de774ae29e768c890304c799cba322c2315f44f78691dcead442661 WatchSource:0}: Error finding container af15560b9de774ae29e768c890304c799cba322c2315f44f78691dcead442661: Status 404 returned error can't find the container with id af15560b9de774ae29e768c890304c799cba322c2315f44f78691dcead442661
Jul 20 21:29:07.567501 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:07.567459    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v" event={"ID":"64e6a540-81a0-4b44-b6b0-79b6e7c4a849","Type":"ContainerStarted","Data":"406df89651082763657660c23b05d13f3458d2510272cb6f8ed26fcf9827381d"}
Jul 20 21:29:07.567885 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:07.567513    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v" event={"ID":"64e6a540-81a0-4b44-b6b0-79b6e7c4a849","Type":"ContainerStarted","Data":"af15560b9de774ae29e768c890304c799cba322c2315f44f78691dcead442661"}
Jul 20 21:29:07.586666 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:07.586610    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v" podStartSLOduration=1.5865958089999999 podStartE2EDuration="1.586595809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:29:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:29:07.584530028 +0000 UTC m=+1588.488519606" watchObservedRunningTime="2026-07-20 21:29:07.586595809 +0000 UTC m=+1588.490585386"
Jul 20 21:29:16.615122 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:16.615089    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:16.615632 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:16.615137    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:16.627592 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:16.627567    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:17.340102 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:17.337744    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:29:17.624001 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:17.623915    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:40.946928 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:40.946849    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"]
Jul 20 21:29:41.599512 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:41.599475    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:29:55.966594 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:55.966531    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v" podUID="64e6a540-81a0-4b44-b6b0-79b6e7c4a849" containerName="main" containerID="cri-o://406df89651082763657660c23b05d13f3458d2510272cb6f8ed26fcf9827381d" gracePeriod=45
Jul 20 21:29:56.214691 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.214667    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:56.362458 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.362426    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-tls-certs\") pod \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\" (UID: \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\") "
Jul 20 21:29:56.362635 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.362480    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-dshm\") pod \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\" (UID: \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\") "
Jul 20 21:29:56.362635 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.362529    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-model-cache\") pod \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\" (UID: \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\") "
Jul 20 21:29:56.362635 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.362580    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-home\") pod \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\" (UID: \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\") "
Jul 20 21:29:56.362635 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.362615    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-tmp-dir\") pod \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\" (UID: \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\") "
Jul 20 21:29:56.362848 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.362664    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfhgs\" (UniqueName: \"kubernetes.io/projected/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-kube-api-access-kfhgs\") pod \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\" (UID: \"64e6a540-81a0-4b44-b6b0-79b6e7c4a849\") "
Jul 20 21:29:56.362905 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.362843    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-model-cache" (OuterVolumeSpecName: "model-cache") pod "64e6a540-81a0-4b44-b6b0-79b6e7c4a849" (UID: "64e6a540-81a0-4b44-b6b0-79b6e7c4a849"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:29:56.362905 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.362873    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-home" (OuterVolumeSpecName: "home") pod "64e6a540-81a0-4b44-b6b0-79b6e7c4a849" (UID: "64e6a540-81a0-4b44-b6b0-79b6e7c4a849"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:29:56.362905 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.362884    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "64e6a540-81a0-4b44-b6b0-79b6e7c4a849" (UID: "64e6a540-81a0-4b44-b6b0-79b6e7c4a849"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:29:56.363019 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.362956    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-model-cache\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:29:56.363019 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.362973    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-home\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:29:56.363019 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.362987    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-tmp-dir\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:29:56.364802 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.364777    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "64e6a540-81a0-4b44-b6b0-79b6e7c4a849" (UID: "64e6a540-81a0-4b44-b6b0-79b6e7c4a849"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 21:29:56.364898 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.364793    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-kube-api-access-kfhgs" (OuterVolumeSpecName: "kube-api-access-kfhgs") pod "64e6a540-81a0-4b44-b6b0-79b6e7c4a849" (UID: "64e6a540-81a0-4b44-b6b0-79b6e7c4a849"). InnerVolumeSpecName "kube-api-access-kfhgs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 21:29:56.364898 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.364782    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-dshm" (OuterVolumeSpecName: "dshm") pod "64e6a540-81a0-4b44-b6b0-79b6e7c4a849" (UID: "64e6a540-81a0-4b44-b6b0-79b6e7c4a849"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:29:56.464120 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.464088    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfhgs\" (UniqueName: \"kubernetes.io/projected/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-kube-api-access-kfhgs\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:29:56.464120 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.464117    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-tls-certs\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:29:56.464120 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.464127    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64e6a540-81a0-4b44-b6b0-79b6e7c4a849-dshm\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:29:56.788502 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.788464    2561 generic.go:356] "Generic (PLEG): container finished" podID="64e6a540-81a0-4b44-b6b0-79b6e7c4a849" containerID="406df89651082763657660c23b05d13f3458d2510272cb6f8ed26fcf9827381d" exitCode=0
Jul 20 21:29:56.788661 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.788501    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"
Jul 20 21:29:56.788661 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.788556    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v" event={"ID":"64e6a540-81a0-4b44-b6b0-79b6e7c4a849","Type":"ContainerDied","Data":"406df89651082763657660c23b05d13f3458d2510272cb6f8ed26fcf9827381d"}
Jul 20 21:29:56.788661 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.788589    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v" event={"ID":"64e6a540-81a0-4b44-b6b0-79b6e7c4a849","Type":"ContainerDied","Data":"af15560b9de774ae29e768c890304c799cba322c2315f44f78691dcead442661"}
Jul 20 21:29:56.788661 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.788605    2561 scope.go:117] "RemoveContainer" containerID="406df89651082763657660c23b05d13f3458d2510272cb6f8ed26fcf9827381d"
Jul 20 21:29:56.797603 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.797586    2561 scope.go:117] "RemoveContainer" containerID="406df89651082763657660c23b05d13f3458d2510272cb6f8ed26fcf9827381d"
Jul 20 21:29:56.797846 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:29:56.797830    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"406df89651082763657660c23b05d13f3458d2510272cb6f8ed26fcf9827381d\": container with ID starting with 406df89651082763657660c23b05d13f3458d2510272cb6f8ed26fcf9827381d not found: ID does not exist" containerID="406df89651082763657660c23b05d13f3458d2510272cb6f8ed26fcf9827381d"
Jul 20 21:29:56.797886 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.797859    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"406df89651082763657660c23b05d13f3458d2510272cb6f8ed26fcf9827381d"} err="failed to get container status \"406df89651082763657660c23b05d13f3458d2510272cb6f8ed26fcf9827381d\": rpc error: code = NotFound desc = could not find container \"406df89651082763657660c23b05d13f3458d2510272cb6f8ed26fcf9827381d\": container with ID starting with 406df89651082763657660c23b05d13f3458d2510272cb6f8ed26fcf9827381d not found: ID does not exist"
Jul 20 21:29:56.815487 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.815467    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"]
Jul 20 21:29:56.830687 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:56.830666    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-690af3af-kserve-6b77f64b9f-gfv8v"]
Jul 20 21:29:57.712687 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:57.712655    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:29:57.725627 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:29:57.725598    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64e6a540-81a0-4b44-b6b0-79b6e7c4a849" path="/var/lib/kubelet/pods/64e6a540-81a0-4b44-b6b0-79b6e7c4a849/volumes"
Jul 20 21:30:00.299980 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:30:00.299944    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743050-67qjh"]
Jul 20 21:30:00.300502 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:30:00.300481    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64e6a540-81a0-4b44-b6b0-79b6e7c4a849" containerName="main"
Jul 20 21:30:00.300552 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:30:00.300508    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="64e6a540-81a0-4b44-b6b0-79b6e7c4a849" containerName="main"
Jul 20 21:30:00.300635 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:30:00.300623    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="64e6a540-81a0-4b44-b6b0-79b6e7c4a849" containerName="main"
Jul 20 21:30:00.305169 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:30:00.305148    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743050-67qjh"]
Jul 20 21:30:00.305294 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:30:00.305275    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743050-67qjh"
Jul 20 21:30:00.312954 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:30:00.312925    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qr6f9\""
Jul 20 21:30:00.316589 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:30:00.316570    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743050-67qjh"
Jul 20 21:30:00.510448 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:30:00.510418    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743050-67qjh"]
Jul 20 21:30:00.513791 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:30:00.513758    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1045438e_4f31_4a0d_a478_30e35b460321.slice/crio-1076b380eab632efc187d18356904047830f58644aca860dc46809fe901d5f57 WatchSource:0}: Error finding container 1076b380eab632efc187d18356904047830f58644aca860dc46809fe901d5f57: Status 404 returned error can't find the container with id 1076b380eab632efc187d18356904047830f58644aca860dc46809fe901d5f57
Jul 20 21:30:00.515568 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:30:00.515553    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 21:30:00.806861 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:30:00.806826    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743050-67qjh" event={"ID":"1045438e-4f31-4a0d-a478-30e35b460321","Type":"ContainerStarted","Data":"1076b380eab632efc187d18356904047830f58644aca860dc46809fe901d5f57"}
Jul 20 21:30:03.824801 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:30:03.824764    2561 generic.go:356] "Generic (PLEG): container finished" podID="1045438e-4f31-4a0d-a478-30e35b460321" containerID="87d2249623fd6bc1701596abfb9ad82a4a137a36763898e5aaf1f6c3584b0450" exitCode=0
Jul 20 21:30:03.825215 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:30:03.824853    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743050-67qjh" event={"ID":"1045438e-4f31-4a0d-a478-30e35b460321","Type":"ContainerDied","Data":"87d2249623fd6bc1701596abfb9ad82a4a137a36763898e5aaf1f6c3584b0450"}
Jul 20 21:30:04.953482 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:30:04.953457    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743050-67qjh"
Jul 20 21:30:05.834537 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:30:05.834505    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743050-67qjh"
Jul 20 21:30:05.834705 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:30:05.834530    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743050-67qjh" event={"ID":"1045438e-4f31-4a0d-a478-30e35b460321","Type":"ContainerDied","Data":"1076b380eab632efc187d18356904047830f58644aca860dc46809fe901d5f57"}
Jul 20 21:30:05.834705 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:30:05.834582    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1076b380eab632efc187d18356904047830f58644aca860dc46809fe901d5f57"
Jul 20 21:30:06.951751 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:30:06.951709    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:30:18.714318 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:30:18.714279    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:30:33.852814 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:30:33.852781    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:30:43.931044 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:30:43.931010    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:30:58.605984 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:30:58.605940    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:31:09.695972 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:31:09.695894    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:31:28.899226 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:31:28.899189    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:31:46.617573 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:31:46.617538    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:31:55.559647 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:31:55.559612    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:32:07.618909 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:32:07.618873    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:32:24.215026 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:32:24.214987    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:32:35.128286 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:32:35.128206    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:32:40.451984 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:32:40.451884    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/1.log"
Jul 20 21:32:40.459128 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:32:40.456058    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/1.log"
Jul 20 21:32:47.885793 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:32:47.885756    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:32:57.027424 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:32:57.027390    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:33:07.589794 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:07.589759    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:33:20.907515 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:20.907480    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:33:37.193578 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:37.193541    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:33:54.976909 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:54.976876    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9d06801-z2f89"]
Jul 20 21:33:54.977300 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:54.977234    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1045438e-4f31-4a0d-a478-30e35b460321" containerName="cleanup"
Jul 20 21:33:54.977300 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:54.977249    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1045438e-4f31-4a0d-a478-30e35b460321" containerName="cleanup"
Jul 20 21:33:54.977376 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:54.977320    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1045438e-4f31-4a0d-a478-30e35b460321" containerName="cleanup"
Jul 20 21:33:54.980031 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:54.980014    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9d06801-z2f89"
Jul 20 21:33:54.982854 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:54.982830    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 21:33:54.987014 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:54.986995    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qr6f9\""
Jul 20 21:33:54.994253 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:54.994231    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9d06801-z2f89"]
Jul 20 21:33:55.060317 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:55.060283    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89ebe9d3-c62b-45d3-abda-459697409f7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c9d06801-z2f89\" (UID: \"89ebe9d3-c62b-45d3-abda-459697409f7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9d06801-z2f89"
Jul 20 21:33:55.161664 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:55.161623    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89ebe9d3-c62b-45d3-abda-459697409f7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c9d06801-z2f89\" (UID: \"89ebe9d3-c62b-45d3-abda-459697409f7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9d06801-z2f89"
Jul 20 21:33:55.162380 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:55.162353    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89ebe9d3-c62b-45d3-abda-459697409f7d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c9d06801-z2f89\" (UID: \"89ebe9d3-c62b-45d3-abda-459697409f7d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9d06801-z2f89"
Jul 20 21:33:55.289899 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:55.289840    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9d06801-z2f89"
Jul 20 21:33:55.447587 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:55.447559    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9d06801-z2f89"]
Jul 20 21:33:55.449472 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:33:55.449434    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89ebe9d3_c62b_45d3_abda_459697409f7d.slice/crio-c361bf870a9b950c7dfdccf487b03ef68a9062acd39a42e5e39e49024346689c WatchSource:0}: Error finding container c361bf870a9b950c7dfdccf487b03ef68a9062acd39a42e5e39e49024346689c: Status 404 returned error can't find the container with id c361bf870a9b950c7dfdccf487b03ef68a9062acd39a42e5e39e49024346689c
Jul 20 21:33:55.887944 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:55.887905    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9d06801-z2f89" event={"ID":"89ebe9d3-c62b-45d3-abda-459697409f7d","Type":"ContainerStarted","Data":"1faf8ecf3b7559d019308d747dd345beafc1933783ebcc5da107dace05af1767"}
Jul 20 21:33:55.888131 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:55.887961    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9d06801-z2f89" event={"ID":"89ebe9d3-c62b-45d3-abda-459697409f7d","Type":"ContainerStarted","Data":"c361bf870a9b950c7dfdccf487b03ef68a9062acd39a42e5e39e49024346689c"}
Jul 20 21:33:56.893975 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:56.893939    2561 generic.go:356] "Generic (PLEG): container finished" podID="89ebe9d3-c62b-45d3-abda-459697409f7d" containerID="1faf8ecf3b7559d019308d747dd345beafc1933783ebcc5da107dace05af1767" exitCode=0
Jul 20 21:33:56.894413 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:56.894026    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9d06801-z2f89" event={"ID":"89ebe9d3-c62b-45d3-abda-459697409f7d","Type":"ContainerDied","Data":"1faf8ecf3b7559d019308d747dd345beafc1933783ebcc5da107dace05af1767"}
Jul 20 21:33:58.028885 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:58.028863    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9d06801-z2f89"
Jul 20 21:33:58.088634 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:58.088603    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89ebe9d3-c62b-45d3-abda-459697409f7d-maas-api-service-ca\") pod \"89ebe9d3-c62b-45d3-abda-459697409f7d\" (UID: \"89ebe9d3-c62b-45d3-abda-459697409f7d\") "
Jul 20 21:33:58.088965 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:58.088941    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89ebe9d3-c62b-45d3-abda-459697409f7d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "89ebe9d3-c62b-45d3-abda-459697409f7d" (UID: "89ebe9d3-c62b-45d3-abda-459697409f7d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 21:33:58.189600 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:58.189526    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89ebe9d3-c62b-45d3-abda-459697409f7d-maas-api-service-ca\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:33:58.904447 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:58.904409    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9d06801-z2f89" event={"ID":"89ebe9d3-c62b-45d3-abda-459697409f7d","Type":"ContainerDied","Data":"c361bf870a9b950c7dfdccf487b03ef68a9062acd39a42e5e39e49024346689c"}
Jul 20 21:33:58.904651 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:58.904468    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c361bf870a9b950c7dfdccf487b03ef68a9062acd39a42e5e39e49024346689c"
Jul 20 21:33:58.904651 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:33:58.904426    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9d06801-z2f89"
Jul 20 21:34:08.502929 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:08.502848    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:34:10.098933 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:10.098896    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9d06801-z2f89"]
Jul 20 21:34:10.108275 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:10.108247    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c9d06801-z2f89"]
Jul 20 21:34:11.380581 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.380545    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"]
Jul 20 21:34:11.380961 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.380906    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89ebe9d3-c62b-45d3-abda-459697409f7d" containerName="revoke-keys"
Jul 20 21:34:11.380961 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.380917    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="89ebe9d3-c62b-45d3-abda-459697409f7d" containerName="revoke-keys"
Jul 20 21:34:11.381028 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.380978    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="89ebe9d3-c62b-45d3-abda-459697409f7d" containerName="revoke-keys"
Jul 20 21:34:11.384179 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.384164    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.387360 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.387336    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-cffb604b-gw-openshift-default-dockercfg-vskmz\""
Jul 20 21:34:11.405312 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.405285    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"]
Jul 20 21:34:11.518502 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.518467    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-workload-certs\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.518502 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.518504    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-istio-envoy\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.518702 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.518523    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-istio-data\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.518702 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.518650    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcxss\" (UniqueName: \"kubernetes.io/projected/ba1a973c-3a18-4f79-9d80-092cce70c69d-kube-api-access-fcxss\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.518702 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.518688    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-credential-socket\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.518824 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.518710    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba1a973c-3a18-4f79-9d80-092cce70c69d-istio-podinfo\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.518824 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.518773    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba1a973c-3a18-4f79-9d80-092cce70c69d-istio-token\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.518824 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.518801    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-workload-socket\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.518824 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.518821    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba1a973c-3a18-4f79-9d80-092cce70c69d-istiod-ca-cert\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.619329 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.619289    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-workload-certs\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.619329 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.619326    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-istio-envoy\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.619580 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.619435    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-istio-data\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.619580 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.619533    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcxss\" (UniqueName: \"kubernetes.io/projected/ba1a973c-3a18-4f79-9d80-092cce70c69d-kube-api-access-fcxss\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.619580 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.619562    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-credential-socket\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.619740 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.619585    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba1a973c-3a18-4f79-9d80-092cce70c69d-istio-podinfo\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.619740 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.619643    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba1a973c-3a18-4f79-9d80-092cce70c69d-istio-token\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.619740 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.619676    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-workload-socket\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.619740 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.619703    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba1a973c-3a18-4f79-9d80-092cce70c69d-istiod-ca-cert\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.619940 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.619762    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-workload-certs\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.619940 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.619814    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-istio-data\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.620043 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.619973    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-credential-socket\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.620196 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.620172    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-workload-socket\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.620395 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.620370    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba1a973c-3a18-4f79-9d80-092cce70c69d-istiod-ca-cert\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.621541 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.621521    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-istio-envoy\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.622185 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.622168    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba1a973c-3a18-4f79-9d80-092cce70c69d-istio-podinfo\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.630505 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.630484    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcxss\" (UniqueName: \"kubernetes.io/projected/ba1a973c-3a18-4f79-9d80-092cce70c69d-kube-api-access-fcxss\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.630653 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.630609    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba1a973c-3a18-4f79-9d80-092cce70c69d-istio-token\") pod \"e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") " pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.695064 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.695024    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:11.726254 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.726226    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89ebe9d3-c62b-45d3-abda-459697409f7d" path="/var/lib/kubelet/pods/89ebe9d3-c62b-45d3-abda-459697409f7d/volumes"
Jul 20 21:34:11.831404 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.831379    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"]
Jul 20 21:34:11.834248 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:34:11.834216    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba1a973c_3a18_4f79_9d80_092cce70c69d.slice/crio-deaae23cf96ed7e79d25e91374868f345457b662f3dedad9998acb21ddefdb58 WatchSource:0}: Error finding container deaae23cf96ed7e79d25e91374868f345457b662f3dedad9998acb21ddefdb58: Status 404 returned error can't find the container with id deaae23cf96ed7e79d25e91374868f345457b662f3dedad9998acb21ddefdb58
Jul 20 21:34:11.836856 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.836826    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:34:11.836942 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.836892    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:34:11.836942 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.836922    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:34:11.966344 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.966302    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt" event={"ID":"ba1a973c-3a18-4f79-9d80-092cce70c69d","Type":"ContainerStarted","Data":"2519525ba6df378989f8013233b069f5fdff9f422f5604fd8b2da761867babc9"}
Jul 20 21:34:11.966507 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.966369    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt" event={"ID":"ba1a973c-3a18-4f79-9d80-092cce70c69d","Type":"ContainerStarted","Data":"deaae23cf96ed7e79d25e91374868f345457b662f3dedad9998acb21ddefdb58"}
Jul 20 21:34:11.998957 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:11.998871    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt" podStartSLOduration=0.998849487 podStartE2EDuration="998.849487ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:34:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:34:11.997999422 +0000 UTC m=+1892.901988992" watchObservedRunningTime="2026-07-20 21:34:11.998849487 +0000 UTC m=+1892.902839069"
Jul 20 21:34:12.696069 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:12.696034    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:12.700834 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:12.700810    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:12.971529 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:12.971439    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:12.972520 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:12.972501    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:20.612027 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:20.611990    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:34:28.186471 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:28.186190    2561 patch_prober.go:28] interesting pod/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": read tcp 10.133.0.2:41786->10.133.0.36:15021: read: connection reset by peer" start-of-body=
Jul 20 21:34:28.186471 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:28.186251    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt" podUID="ba1a973c-3a18-4f79-9d80-092cce70c69d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": read tcp 10.133.0.2:41786->10.133.0.36:15021: read: connection reset by peer"
Jul 20 21:34:29.049832 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:29.049801    2561 generic.go:356] "Generic (PLEG): container finished" podID="ba1a973c-3a18-4f79-9d80-092cce70c69d" containerID="2519525ba6df378989f8013233b069f5fdff9f422f5604fd8b2da761867babc9" exitCode=0
Jul 20 21:34:29.050018 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:29.049883    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt" event={"ID":"ba1a973c-3a18-4f79-9d80-092cce70c69d","Type":"ContainerDied","Data":"2519525ba6df378989f8013233b069f5fdff9f422f5604fd8b2da761867babc9"}
Jul 20 21:34:29.055684 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:29.050762    2561 scope.go:117] "RemoveContainer" containerID="2519525ba6df378989f8013233b069f5fdff9f422f5604fd8b2da761867babc9"
Jul 20 21:34:29.055684 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:29.055177    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:34:29.055684 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:29.055317    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:34:29.055684 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:29.055414    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:34:30.060003 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:30.059957    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt" event={"ID":"ba1a973c-3a18-4f79-9d80-092cce70c69d","Type":"ContainerStarted","Data":"b5380554fbbfc133fc34301c3a6cfee0a7d1e3d59f6ff3f7150a81be09a10052"}
Jul 20 21:34:30.695381 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:30.695346    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:30.695595 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:30.695500    2561 patch_prober.go:28] interesting pod/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 20 21:34:30.695595 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:30.695578    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt" podUID="ba1a973c-3a18-4f79-9d80-092cce70c69d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 20 21:34:31.695954 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:31.695919    2561 patch_prober.go:28] interesting pod/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 20 21:34:31.696353 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:31.695968    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt" podUID="ba1a973c-3a18-4f79-9d80-092cce70c69d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 20 21:34:32.695553 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:32.695516    2561 patch_prober.go:28] interesting pod/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 20 21:34:32.695742 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:32.695576    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt" podUID="ba1a973c-3a18-4f79-9d80-092cce70c69d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 20 21:34:33.695568 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:33.695528    2561 patch_prober.go:28] interesting pod/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 20 21:34:33.695970 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:33.695585    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt" podUID="ba1a973c-3a18-4f79-9d80-092cce70c69d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 20 21:34:34.698817 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:34.698789    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:34.699292 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:34.699163    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:34.699710 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:34.699693    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:38.631049 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:38.631007    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-cffb604b-69bzh"]
Jul 20 21:34:38.633504 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:38.633488    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-cffb604b-69bzh"
Jul 20 21:34:38.636321 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:38.636294    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qr6f9\""
Jul 20 21:34:38.636446 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:38.636343    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 21:34:38.641976 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:38.641950    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-cffb604b-69bzh"]
Jul 20 21:34:38.663818 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:38.663786    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fddd397-a836-49e1-8697-eb14c15020f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-cffb604b-69bzh\" (UID: \"7fddd397-a836-49e1-8697-eb14c15020f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-cffb604b-69bzh"
Jul 20 21:34:38.764822 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:38.764779    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fddd397-a836-49e1-8697-eb14c15020f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-cffb604b-69bzh\" (UID: \"7fddd397-a836-49e1-8697-eb14c15020f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-cffb604b-69bzh"
Jul 20 21:34:38.765457 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:38.765436    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fddd397-a836-49e1-8697-eb14c15020f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-cffb604b-69bzh\" (UID: \"7fddd397-a836-49e1-8697-eb14c15020f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-cffb604b-69bzh"
Jul 20 21:34:38.945871 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:38.945789    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-cffb604b-69bzh"
Jul 20 21:34:39.079965 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:39.079928    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-cffb604b-69bzh"]
Jul 20 21:34:39.087786 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:34:39.087750    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fddd397_a836_49e1_8697_eb14c15020f0.slice/crio-c5b8e7e79571cd46511eaa1b9d798c58913156ba3c9095b35da2f729d6e4d375 WatchSource:0}: Error finding container c5b8e7e79571cd46511eaa1b9d798c58913156ba3c9095b35da2f729d6e4d375: Status 404 returned error can't find the container with id c5b8e7e79571cd46511eaa1b9d798c58913156ba3c9095b35da2f729d6e4d375
Jul 20 21:34:39.107245 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:39.107208    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-cffb604b-69bzh" event={"ID":"7fddd397-a836-49e1-8697-eb14c15020f0","Type":"ContainerStarted","Data":"c5b8e7e79571cd46511eaa1b9d798c58913156ba3c9095b35da2f729d6e4d375"}
Jul 20 21:34:40.113362 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:40.113326    2561 generic.go:356] "Generic (PLEG): container finished" podID="7fddd397-a836-49e1-8697-eb14c15020f0" containerID="cf33d3948aab3532e43e30044577d2f7f0eb210b4f59e2b015ba8ea5c37968c8" exitCode=0
Jul 20 21:34:40.113814 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:40.113415    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-cffb604b-69bzh" event={"ID":"7fddd397-a836-49e1-8697-eb14c15020f0","Type":"ContainerDied","Data":"cf33d3948aab3532e43e30044577d2f7f0eb210b4f59e2b015ba8ea5c37968c8"}
Jul 20 21:34:41.254856 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:41.254834    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-cffb604b-69bzh"
Jul 20 21:34:41.287850 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:41.287821    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fddd397-a836-49e1-8697-eb14c15020f0-maas-api-service-ca\") pod \"7fddd397-a836-49e1-8697-eb14c15020f0\" (UID: \"7fddd397-a836-49e1-8697-eb14c15020f0\") "
Jul 20 21:34:41.288168 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:41.288147    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fddd397-a836-49e1-8697-eb14c15020f0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7fddd397-a836-49e1-8697-eb14c15020f0" (UID: "7fddd397-a836-49e1-8697-eb14c15020f0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 21:34:41.389383 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:41.389297    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fddd397-a836-49e1-8697-eb14c15020f0-maas-api-service-ca\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:34:42.124445 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:42.124411    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-cffb604b-69bzh" event={"ID":"7fddd397-a836-49e1-8697-eb14c15020f0","Type":"ContainerDied","Data":"c5b8e7e79571cd46511eaa1b9d798c58913156ba3c9095b35da2f729d6e4d375"}
Jul 20 21:34:42.124445 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:42.124447    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-cffb604b-69bzh"
Jul 20 21:34:42.124647 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:42.124459    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c5b8e7e79571cd46511eaa1b9d798c58913156ba3c9095b35da2f729d6e4d375"
Jul 20 21:34:52.284607 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:52.284571    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:34:53.715264 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:53.715225    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-cffb604b-69bzh"]
Jul 20 21:34:53.718597 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:53.718574    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-cffb604b-69bzh"]
Jul 20 21:34:53.726641 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:53.726614    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fddd397-a836-49e1-8697-eb14c15020f0" path="/var/lib/kubelet/pods/7fddd397-a836-49e1-8697-eb14c15020f0/volumes"
Jul 20 21:34:53.852677 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:53.852648    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"]
Jul 20 21:34:53.852876 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:53.852851    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt" podUID="ba1a973c-3a18-4f79-9d80-092cce70c69d" containerName="istio-proxy" containerID="cri-o://b5380554fbbfc133fc34301c3a6cfee0a7d1e3d59f6ff3f7150a81be09a10052" gracePeriod=30
Jul 20 21:34:59.105120 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.105068    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:59.205602 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.205503    2561 generic.go:356] "Generic (PLEG): container finished" podID="ba1a973c-3a18-4f79-9d80-092cce70c69d" containerID="b5380554fbbfc133fc34301c3a6cfee0a7d1e3d59f6ff3f7150a81be09a10052" exitCode=0
Jul 20 21:34:59.205602 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.205560    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"
Jul 20 21:34:59.205602 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.205586    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt" event={"ID":"ba1a973c-3a18-4f79-9d80-092cce70c69d","Type":"ContainerDied","Data":"b5380554fbbfc133fc34301c3a6cfee0a7d1e3d59f6ff3f7150a81be09a10052"}
Jul 20 21:34:59.205875 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.205637    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt" event={"ID":"ba1a973c-3a18-4f79-9d80-092cce70c69d","Type":"ContainerDied","Data":"deaae23cf96ed7e79d25e91374868f345457b662f3dedad9998acb21ddefdb58"}
Jul 20 21:34:59.205875 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.205662    2561 scope.go:117] "RemoveContainer" containerID="b5380554fbbfc133fc34301c3a6cfee0a7d1e3d59f6ff3f7150a81be09a10052"
Jul 20 21:34:59.214157 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.214135    2561 scope.go:117] "RemoveContainer" containerID="2519525ba6df378989f8013233b069f5fdff9f422f5604fd8b2da761867babc9"
Jul 20 21:34:59.221655 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.221640    2561 scope.go:117] "RemoveContainer" containerID="b5380554fbbfc133fc34301c3a6cfee0a7d1e3d59f6ff3f7150a81be09a10052"
Jul 20 21:34:59.221907 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:34:59.221889    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5380554fbbfc133fc34301c3a6cfee0a7d1e3d59f6ff3f7150a81be09a10052\": container with ID starting with b5380554fbbfc133fc34301c3a6cfee0a7d1e3d59f6ff3f7150a81be09a10052 not found: ID does not exist" containerID="b5380554fbbfc133fc34301c3a6cfee0a7d1e3d59f6ff3f7150a81be09a10052"
Jul 20 21:34:59.221973 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.221920    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5380554fbbfc133fc34301c3a6cfee0a7d1e3d59f6ff3f7150a81be09a10052"} err="failed to get container status \"b5380554fbbfc133fc34301c3a6cfee0a7d1e3d59f6ff3f7150a81be09a10052\": rpc error: code = NotFound desc = could not find container \"b5380554fbbfc133fc34301c3a6cfee0a7d1e3d59f6ff3f7150a81be09a10052\": container with ID starting with b5380554fbbfc133fc34301c3a6cfee0a7d1e3d59f6ff3f7150a81be09a10052 not found: ID does not exist"
Jul 20 21:34:59.221973 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.221946    2561 scope.go:117] "RemoveContainer" containerID="2519525ba6df378989f8013233b069f5fdff9f422f5604fd8b2da761867babc9"
Jul 20 21:34:59.222208 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:34:59.222190    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2519525ba6df378989f8013233b069f5fdff9f422f5604fd8b2da761867babc9\": container with ID starting with 2519525ba6df378989f8013233b069f5fdff9f422f5604fd8b2da761867babc9 not found: ID does not exist" containerID="2519525ba6df378989f8013233b069f5fdff9f422f5604fd8b2da761867babc9"
Jul 20 21:34:59.222254 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.222215    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2519525ba6df378989f8013233b069f5fdff9f422f5604fd8b2da761867babc9"} err="failed to get container status \"2519525ba6df378989f8013233b069f5fdff9f422f5604fd8b2da761867babc9\": rpc error: code = NotFound desc = could not find container \"2519525ba6df378989f8013233b069f5fdff9f422f5604fd8b2da761867babc9\": container with ID starting with 2519525ba6df378989f8013233b069f5fdff9f422f5604fd8b2da761867babc9 not found: ID does not exist"
Jul 20 21:34:59.259579 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.259538    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba1a973c-3a18-4f79-9d80-092cce70c69d-istio-podinfo\") pod \"ba1a973c-3a18-4f79-9d80-092cce70c69d\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") "
Jul 20 21:34:59.259778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.259603    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-workload-socket\") pod \"ba1a973c-3a18-4f79-9d80-092cce70c69d\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") "
Jul 20 21:34:59.259778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.259629    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-istio-envoy\") pod \"ba1a973c-3a18-4f79-9d80-092cce70c69d\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") "
Jul 20 21:34:59.259778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.259655    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-istio-data\") pod \"ba1a973c-3a18-4f79-9d80-092cce70c69d\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") "
Jul 20 21:34:59.259778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.259680    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba1a973c-3a18-4f79-9d80-092cce70c69d-istiod-ca-cert\") pod \"ba1a973c-3a18-4f79-9d80-092cce70c69d\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") "
Jul 20 21:34:59.259778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.259741    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcxss\" (UniqueName: \"kubernetes.io/projected/ba1a973c-3a18-4f79-9d80-092cce70c69d-kube-api-access-fcxss\") pod \"ba1a973c-3a18-4f79-9d80-092cce70c69d\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") "
Jul 20 21:34:59.259778 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.259765    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-credential-socket\") pod \"ba1a973c-3a18-4f79-9d80-092cce70c69d\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") "
Jul 20 21:34:59.260107 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.259803    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba1a973c-3a18-4f79-9d80-092cce70c69d-istio-token\") pod \"ba1a973c-3a18-4f79-9d80-092cce70c69d\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") "
Jul 20 21:34:59.260107 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.259834    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-workload-certs\") pod \"ba1a973c-3a18-4f79-9d80-092cce70c69d\" (UID: \"ba1a973c-3a18-4f79-9d80-092cce70c69d\") "
Jul 20 21:34:59.260107 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.259914    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ba1a973c-3a18-4f79-9d80-092cce70c69d" (UID: "ba1a973c-3a18-4f79-9d80-092cce70c69d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:34:59.260266 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.260106    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ba1a973c-3a18-4f79-9d80-092cce70c69d" (UID: "ba1a973c-3a18-4f79-9d80-092cce70c69d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:34:59.260266 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.260200    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba1a973c-3a18-4f79-9d80-092cce70c69d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ba1a973c-3a18-4f79-9d80-092cce70c69d" (UID: "ba1a973c-3a18-4f79-9d80-092cce70c69d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 21:34:59.260266 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.260222    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-workload-socket\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:34:59.260266 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.260242    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-credential-socket\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:34:59.260468 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.260314    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ba1a973c-3a18-4f79-9d80-092cce70c69d" (UID: "ba1a973c-3a18-4f79-9d80-092cce70c69d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:34:59.260852 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.260828    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-istio-data" (OuterVolumeSpecName: "istio-data") pod "ba1a973c-3a18-4f79-9d80-092cce70c69d" (UID: "ba1a973c-3a18-4f79-9d80-092cce70c69d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:34:59.262063 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.262033    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ba1a973c-3a18-4f79-9d80-092cce70c69d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ba1a973c-3a18-4f79-9d80-092cce70c69d" (UID: "ba1a973c-3a18-4f79-9d80-092cce70c69d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 21:34:59.262194 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.262069    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba1a973c-3a18-4f79-9d80-092cce70c69d-istio-token" (OuterVolumeSpecName: "istio-token") pod "ba1a973c-3a18-4f79-9d80-092cce70c69d" (UID: "ba1a973c-3a18-4f79-9d80-092cce70c69d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 21:34:59.262289 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.262270    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ba1a973c-3a18-4f79-9d80-092cce70c69d" (UID: "ba1a973c-3a18-4f79-9d80-092cce70c69d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:34:59.262401 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.262377    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba1a973c-3a18-4f79-9d80-092cce70c69d-kube-api-access-fcxss" (OuterVolumeSpecName: "kube-api-access-fcxss") pod "ba1a973c-3a18-4f79-9d80-092cce70c69d" (UID: "ba1a973c-3a18-4f79-9d80-092cce70c69d"). InnerVolumeSpecName "kube-api-access-fcxss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 21:34:59.361684 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.361635    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba1a973c-3a18-4f79-9d80-092cce70c69d-istio-token\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:34:59.361684 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.361687    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-workload-certs\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:34:59.361876 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.361696    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba1a973c-3a18-4f79-9d80-092cce70c69d-istio-podinfo\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:34:59.361876 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.361704    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-istio-envoy\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:34:59.361876 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.361712    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba1a973c-3a18-4f79-9d80-092cce70c69d-istio-data\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:34:59.361876 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.361719    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba1a973c-3a18-4f79-9d80-092cce70c69d-istiod-ca-cert\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:34:59.361876 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.361727    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcxss\" (UniqueName: \"kubernetes.io/projected/ba1a973c-3a18-4f79-9d80-092cce70c69d-kube-api-access-fcxss\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:34:59.531611 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.531578    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"]
Jul 20 21:34:59.536281 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.536254    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-cffb604b-gw-openshift-default-5d4dff9779-5pbbt"]
Jul 20 21:34:59.726503 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:34:59.726473    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba1a973c-3a18-4f79-9d80-092cce70c69d" path="/var/lib/kubelet/pods/ba1a973c-3a18-4f79-9d80-092cce70c69d/volumes"
Jul 20 21:35:05.829804 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:05.829761    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:35:22.673410 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:22.673375    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ce63ea5-r6pdb"]
Jul 20 21:35:22.673918 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:22.673721    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba1a973c-3a18-4f79-9d80-092cce70c69d" containerName="istio-proxy"
Jul 20 21:35:22.673918 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:22.673734    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba1a973c-3a18-4f79-9d80-092cce70c69d" containerName="istio-proxy"
Jul 20 21:35:22.673918 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:22.673751    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fddd397-a836-49e1-8697-eb14c15020f0" containerName="revoke-keys"
Jul 20 21:35:22.673918 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:22.673766    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fddd397-a836-49e1-8697-eb14c15020f0" containerName="revoke-keys"
Jul 20 21:35:22.673918 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:22.673776    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba1a973c-3a18-4f79-9d80-092cce70c69d" containerName="istio-proxy"
Jul 20 21:35:22.673918 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:22.673782    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba1a973c-3a18-4f79-9d80-092cce70c69d" containerName="istio-proxy"
Jul 20 21:35:22.673918 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:22.673832    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fddd397-a836-49e1-8697-eb14c15020f0" containerName="revoke-keys"
Jul 20 21:35:22.673918 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:22.673846    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba1a973c-3a18-4f79-9d80-092cce70c69d" containerName="istio-proxy"
Jul 20 21:35:22.678147 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:22.678118    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ce63ea5-r6pdb"
Jul 20 21:35:22.682379 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:22.682357    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 21:35:22.682558 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:22.682357    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qr6f9\""
Jul 20 21:35:22.683921 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:22.683896    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ce63ea5-r6pdb"]
Jul 20 21:35:22.765252 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:22.765208    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d0a4bf5-fb4f-4205-a6db-9b552cec5020-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9ce63ea5-r6pdb\" (UID: \"6d0a4bf5-fb4f-4205-a6db-9b552cec5020\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ce63ea5-r6pdb"
Jul 20 21:35:22.866145 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:22.866070    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d0a4bf5-fb4f-4205-a6db-9b552cec5020-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9ce63ea5-r6pdb\" (UID: \"6d0a4bf5-fb4f-4205-a6db-9b552cec5020\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ce63ea5-r6pdb"
Jul 20 21:35:22.866740 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:22.866718    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d0a4bf5-fb4f-4205-a6db-9b552cec5020-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9ce63ea5-r6pdb\" (UID: \"6d0a4bf5-fb4f-4205-a6db-9b552cec5020\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ce63ea5-r6pdb"
Jul 20 21:35:22.991415 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:22.991331    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ce63ea5-r6pdb"
Jul 20 21:35:23.120276 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:23.120250    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ce63ea5-r6pdb"]
Jul 20 21:35:23.122206 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:35:23.122183    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d0a4bf5_fb4f_4205_a6db_9b552cec5020.slice/crio-fe08e754090cfd21730dd049c4982878f9d2e75161ef1ebafa3067f31e06d9eb WatchSource:0}: Error finding container fe08e754090cfd21730dd049c4982878f9d2e75161ef1ebafa3067f31e06d9eb: Status 404 returned error can't find the container with id fe08e754090cfd21730dd049c4982878f9d2e75161ef1ebafa3067f31e06d9eb
Jul 20 21:35:23.319772 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:23.319740    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ce63ea5-r6pdb" event={"ID":"6d0a4bf5-fb4f-4205-a6db-9b552cec5020","Type":"ContainerStarted","Data":"fe08e754090cfd21730dd049c4982878f9d2e75161ef1ebafa3067f31e06d9eb"}
Jul 20 21:35:24.326294 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:24.326256    2561 generic.go:356] "Generic (PLEG): container finished" podID="6d0a4bf5-fb4f-4205-a6db-9b552cec5020" containerID="f493ccc8cdc6ad9e0f314f09c717e1efe4a90e0fa3f00c5397b316175628c22d" exitCode=0
Jul 20 21:35:24.326680 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:24.326339    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ce63ea5-r6pdb" event={"ID":"6d0a4bf5-fb4f-4205-a6db-9b552cec5020","Type":"ContainerDied","Data":"f493ccc8cdc6ad9e0f314f09c717e1efe4a90e0fa3f00c5397b316175628c22d"}
Jul 20 21:35:25.462637 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:25.462613    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ce63ea5-r6pdb"
Jul 20 21:35:25.487557 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:25.487528    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d0a4bf5-fb4f-4205-a6db-9b552cec5020-maas-api-service-ca\") pod \"6d0a4bf5-fb4f-4205-a6db-9b552cec5020\" (UID: \"6d0a4bf5-fb4f-4205-a6db-9b552cec5020\") "
Jul 20 21:35:25.487982 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:25.487954    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d0a4bf5-fb4f-4205-a6db-9b552cec5020-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6d0a4bf5-fb4f-4205-a6db-9b552cec5020" (UID: "6d0a4bf5-fb4f-4205-a6db-9b552cec5020"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 21:35:25.588950 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:25.588862    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d0a4bf5-fb4f-4205-a6db-9b552cec5020-maas-api-service-ca\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:35:26.337507 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:26.337471    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ce63ea5-r6pdb"
Jul 20 21:35:26.337507 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:26.337480    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ce63ea5-r6pdb" event={"ID":"6d0a4bf5-fb4f-4205-a6db-9b552cec5020","Type":"ContainerDied","Data":"fe08e754090cfd21730dd049c4982878f9d2e75161ef1ebafa3067f31e06d9eb"}
Jul 20 21:35:26.337718 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:26.337530    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fe08e754090cfd21730dd049c4982878f9d2e75161ef1ebafa3067f31e06d9eb"
Jul 20 21:35:36.078680 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:36.078587    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:35:37.744811 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:37.744773    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ce63ea5-r6pdb"]
Jul 20 21:35:37.746817 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:37.746791    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ce63ea5-r6pdb"]
Jul 20 21:35:39.727014 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:39.726978    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d0a4bf5-fb4f-4205-a6db-9b552cec5020" path="/var/lib/kubelet/pods/6d0a4bf5-fb4f-4205-a6db-9b552cec5020/volumes"
Jul 20 21:35:49.188265 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:35:49.188228    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:36:06.856198 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:06.856160    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5eb07c58-527bb"]
Jul 20 21:36:06.856729 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:06.856497    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d0a4bf5-fb4f-4205-a6db-9b552cec5020" containerName="revoke-keys"
Jul 20 21:36:06.856729 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:06.856507    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d0a4bf5-fb4f-4205-a6db-9b552cec5020" containerName="revoke-keys"
Jul 20 21:36:06.856729 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:06.856566    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba1a973c-3a18-4f79-9d80-092cce70c69d" containerName="istio-proxy"
Jul 20 21:36:06.856729 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:06.856575    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d0a4bf5-fb4f-4205-a6db-9b552cec5020" containerName="revoke-keys"
Jul 20 21:36:06.859487 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:06.859469    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5eb07c58-527bb"
Jul 20 21:36:06.862040 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:06.862017    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 21:36:06.862191 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:06.862143    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qr6f9\""
Jul 20 21:36:06.868094 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:06.868052    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5eb07c58-527bb"]
Jul 20 21:36:06.964174 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:06.964141    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf51f8b6-4c97-442f-aa59-c817810ed31e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5eb07c58-527bb\" (UID: \"cf51f8b6-4c97-442f-aa59-c817810ed31e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5eb07c58-527bb"
Jul 20 21:36:07.065326 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:07.065288    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf51f8b6-4c97-442f-aa59-c817810ed31e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5eb07c58-527bb\" (UID: \"cf51f8b6-4c97-442f-aa59-c817810ed31e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5eb07c58-527bb"
Jul 20 21:36:07.065917 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:07.065899    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf51f8b6-4c97-442f-aa59-c817810ed31e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5eb07c58-527bb\" (UID: \"cf51f8b6-4c97-442f-aa59-c817810ed31e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5eb07c58-527bb"
Jul 20 21:36:07.172053 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:07.171970    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5eb07c58-527bb"
Jul 20 21:36:07.306955 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:07.306846    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5eb07c58-527bb"]
Jul 20 21:36:07.309783 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:36:07.309757    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf51f8b6_4c97_442f_aa59_c817810ed31e.slice/crio-fc8785f03b1910d4ba0e0ce2535da11ca1fb83d9e3936b8b1c699eaf77853766 WatchSource:0}: Error finding container fc8785f03b1910d4ba0e0ce2535da11ca1fb83d9e3936b8b1c699eaf77853766: Status 404 returned error can't find the container with id fc8785f03b1910d4ba0e0ce2535da11ca1fb83d9e3936b8b1c699eaf77853766
Jul 20 21:36:07.538494 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:07.538463    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5eb07c58-527bb" event={"ID":"cf51f8b6-4c97-442f-aa59-c817810ed31e","Type":"ContainerStarted","Data":"fc8785f03b1910d4ba0e0ce2535da11ca1fb83d9e3936b8b1c699eaf77853766"}
Jul 20 21:36:08.544656 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:08.544625    2561 generic.go:356] "Generic (PLEG): container finished" podID="cf51f8b6-4c97-442f-aa59-c817810ed31e" containerID="39206cec519b6de63697f7842853242286f5c8b921b505e2e13cea15112f5386" exitCode=0
Jul 20 21:36:08.545046 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:08.544719    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5eb07c58-527bb" event={"ID":"cf51f8b6-4c97-442f-aa59-c817810ed31e","Type":"ContainerDied","Data":"39206cec519b6de63697f7842853242286f5c8b921b505e2e13cea15112f5386"}
Jul 20 21:36:09.674904 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:09.674882    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5eb07c58-527bb"
Jul 20 21:36:09.789383 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:09.789352    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf51f8b6-4c97-442f-aa59-c817810ed31e-maas-api-service-ca\") pod \"cf51f8b6-4c97-442f-aa59-c817810ed31e\" (UID: \"cf51f8b6-4c97-442f-aa59-c817810ed31e\") "
Jul 20 21:36:09.789747 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:09.789727    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf51f8b6-4c97-442f-aa59-c817810ed31e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cf51f8b6-4c97-442f-aa59-c817810ed31e" (UID: "cf51f8b6-4c97-442f-aa59-c817810ed31e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 21:36:09.890633 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:09.890549    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf51f8b6-4c97-442f-aa59-c817810ed31e-maas-api-service-ca\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:36:10.556198 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:10.556169    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5eb07c58-527bb"
Jul 20 21:36:10.556372 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:10.556168    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5eb07c58-527bb" event={"ID":"cf51f8b6-4c97-442f-aa59-c817810ed31e","Type":"ContainerDied","Data":"fc8785f03b1910d4ba0e0ce2535da11ca1fb83d9e3936b8b1c699eaf77853766"}
Jul 20 21:36:10.556372 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:10.556299    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc8785f03b1910d4ba0e0ce2535da11ca1fb83d9e3936b8b1c699eaf77853766"
Jul 20 21:36:20.200156 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:20.200112    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:36:21.943194 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:21.943157    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5eb07c58-527bb"]
Jul 20 21:36:21.951033 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:21.951005    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5eb07c58-527bb"]
Jul 20 21:36:23.725688 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:36:23.725643    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf51f8b6-4c97-442f-aa59-c817810ed31e" path="/var/lib/kubelet/pods/cf51f8b6-4c97-442f-aa59-c817810ed31e/volumes"
Jul 20 21:37:40.480983 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:37:40.480955    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/1.log"
Jul 20 21:37:40.489032 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:37:40.489009    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/1.log"
Jul 20 21:37:50.392101 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:37:50.392051    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:38:01.707091 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:01.707043    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:38:11.221846 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.221809    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"]
Jul 20 21:38:11.222509 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.222211    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf51f8b6-4c97-442f-aa59-c817810ed31e" containerName="revoke-keys"
Jul 20 21:38:11.222509 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.222222    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf51f8b6-4c97-442f-aa59-c817810ed31e" containerName="revoke-keys"
Jul 20 21:38:11.222509 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.222272    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf51f8b6-4c97-442f-aa59-c817810ed31e" containerName="revoke-keys"
Jul 20 21:38:11.225212 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.225198    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.230062 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.230020    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ff57b586-openshift-default-dockercfg-7gg4f\""
Jul 20 21:38:11.309692 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.309655    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"]
Jul 20 21:38:11.324918 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.324883    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-credential-socket\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.325060 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.324927    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istio-token\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.325060 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.324981    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-workload-socket\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.325060 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.325021    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl82z\" (UniqueName: \"kubernetes.io/projected/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-kube-api-access-dl82z\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.325193 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.325082    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-workload-certs\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.325193 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.325114    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istio-envoy\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.325193 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.325156    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istio-data\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.325287 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.325210    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istiod-ca-cert\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.325287 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.325251    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istio-podinfo\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.425835 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.425795    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-workload-socket\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.425835 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.425842    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl82z\" (UniqueName: \"kubernetes.io/projected/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-kube-api-access-dl82z\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.426114 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.425860    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-workload-certs\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.426114 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.425875    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istio-envoy\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.426114 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.425942    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istio-data\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.426114 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.426011    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istiod-ca-cert\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.426114 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.426097    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istio-podinfo\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.426386 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.426138    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-credential-socket\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.426386 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.426165    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istio-token\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.426386 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.426352    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-workload-socket\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.426386 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.426374    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istio-data\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.426606 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.426441    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-workload-certs\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.426668 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.426644    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-credential-socket\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.426899 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.426873    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istiod-ca-cert\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.428218 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.428169    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istio-envoy\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.428699 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.428680    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istio-podinfo\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.448193 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.448170    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istio-token\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.450211 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.450187    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl82z\" (UniqueName: \"kubernetes.io/projected/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-kube-api-access-dl82z\") pod \"e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") " pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.534485 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.534455    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:11.712521 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.712498    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"]
Jul 20 21:38:11.714917 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:38:11.714892    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f787742_3eed_4ed6_b5cc_cecdca7fcd73.slice/crio-d9b73e9e6b3009762ef0853eb73a52bedcf65a8c9d0dad4a2c5a793c0abfef0d WatchSource:0}: Error finding container d9b73e9e6b3009762ef0853eb73a52bedcf65a8c9d0dad4a2c5a793c0abfef0d: Status 404 returned error can't find the container with id d9b73e9e6b3009762ef0853eb73a52bedcf65a8c9d0dad4a2c5a793c0abfef0d
Jul 20 21:38:11.716911 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.716882    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:38:11.716968 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.716944    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:38:11.717004 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:11.716972    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:38:12.132799 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:12.132760    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd" event={"ID":"5f787742-3eed-4ed6-b5cc-cecdca7fcd73","Type":"ContainerStarted","Data":"946f461493771a1bec21652dda40247a77ee3af59e589b3c247615365d9d3cf0"}
Jul 20 21:38:12.133021 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:12.132821    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd" event={"ID":"5f787742-3eed-4ed6-b5cc-cecdca7fcd73","Type":"ContainerStarted","Data":"d9b73e9e6b3009762ef0853eb73a52bedcf65a8c9d0dad4a2c5a793c0abfef0d"}
Jul 20 21:38:12.181037 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:12.180988    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd" podStartSLOduration=1.180972849 podStartE2EDuration="1.180972849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:38:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:38:12.179499201 +0000 UTC m=+2133.083488780" watchObservedRunningTime="2026-07-20 21:38:12.180972849 +0000 UTC m=+2133.084962425"
Jul 20 21:38:12.535325 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:12.535300    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:12.540138 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:12.540115    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:12.719802 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:12.719765    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:38:13.138293 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:13.138266    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:13.139160 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:13.139136    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:24.688479 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:24.688444    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:38:28.263285 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:28.263232    2561 patch_prober.go:28] interesting pod/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": read tcp 10.133.0.2:41272->10.133.0.40:15021: read: connection reset by peer" start-of-body=
Jul 20 21:38:28.263656 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:28.263309    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd" podUID="5f787742-3eed-4ed6-b5cc-cecdca7fcd73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": read tcp 10.133.0.2:41272->10.133.0.40:15021: read: connection reset by peer"
Jul 20 21:38:29.217741 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:29.217705    2561 generic.go:356] "Generic (PLEG): container finished" podID="5f787742-3eed-4ed6-b5cc-cecdca7fcd73" containerID="946f461493771a1bec21652dda40247a77ee3af59e589b3c247615365d9d3cf0" exitCode=0
Jul 20 21:38:29.217919 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:29.217777    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd" event={"ID":"5f787742-3eed-4ed6-b5cc-cecdca7fcd73","Type":"ContainerDied","Data":"946f461493771a1bec21652dda40247a77ee3af59e589b3c247615365d9d3cf0"}
Jul 20 21:38:29.218217 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:29.218202    2561 scope.go:117] "RemoveContainer" containerID="946f461493771a1bec21652dda40247a77ee3af59e589b3c247615365d9d3cf0"
Jul 20 21:38:29.219190 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:29.219165    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:38:29.219259 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:29.219243    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:38:29.219306 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:29.219291    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:38:30.226664 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:30.226626    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd" event={"ID":"5f787742-3eed-4ed6-b5cc-cecdca7fcd73","Type":"ContainerStarted","Data":"deab21c8b3d4c50065ee324c6ddd67b30a063d57ab1da81414a69b8bac83469f"}
Jul 20 21:38:30.535377 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:30.535349    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:30.535542 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:30.535524    2561 patch_prober.go:28] interesting pod/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Jul 20 21:38:30.535586 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:30.535569    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd" podUID="5f787742-3eed-4ed6-b5cc-cecdca7fcd73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Jul 20 21:38:31.535159 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:31.535116    2561 patch_prober.go:28] interesting pod/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Jul 20 21:38:31.535652 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:31.535190    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd" podUID="5f787742-3eed-4ed6-b5cc-cecdca7fcd73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Jul 20 21:38:32.535683 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:32.535642    2561 patch_prober.go:28] interesting pod/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Jul 20 21:38:32.536170 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:32.535713    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd" podUID="5f787742-3eed-4ed6-b5cc-cecdca7fcd73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Jul 20 21:38:33.535044 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:33.535006    2561 patch_prober.go:28] interesting pod/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Jul 20 21:38:33.535241 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:33.535099    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd" podUID="5f787742-3eed-4ed6-b5cc-cecdca7fcd73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Jul 20 21:38:34.538609 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:34.538573    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:34.539113 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:34.538964    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:34.539544 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:34.539525    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:39.992332 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:39.992304    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:38:48.564119 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:48.564064    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"]
Jul 20 21:38:48.564513 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:48.564334    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd" podUID="5f787742-3eed-4ed6-b5cc-cecdca7fcd73" containerName="istio-proxy" containerID="cri-o://deab21c8b3d4c50065ee324c6ddd67b30a063d57ab1da81414a69b8bac83469f" gracePeriod=30
Jul 20 21:38:49.540894 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:49.540853    2561 patch_prober.go:28] interesting pod/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 20 21:38:49.541112 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:49.540926    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd" podUID="5f787742-3eed-4ed6-b5cc-cecdca7fcd73" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 20 21:38:53.813165 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.813141    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:53.896055 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.895980    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istio-data\") pod \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") "
Jul 20 21:38:53.896055 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.896015    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istiod-ca-cert\") pod \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") "
Jul 20 21:38:53.896055 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.896037    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istio-podinfo\") pod \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") "
Jul 20 21:38:53.896328 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.896087    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istio-token\") pod \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") "
Jul 20 21:38:53.896328 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.896133    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-workload-socket\") pod \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") "
Jul 20 21:38:53.896328 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.896163    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-workload-certs\") pod \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") "
Jul 20 21:38:53.896328 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.896197    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istio-envoy\") pod \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") "
Jul 20 21:38:53.896328 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.896241    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-credential-socket\") pod \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") "
Jul 20 21:38:53.896328 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.896277    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dl82z\" (UniqueName: \"kubernetes.io/projected/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-kube-api-access-dl82z\") pod \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\" (UID: \"5f787742-3eed-4ed6-b5cc-cecdca7fcd73\") "
Jul 20 21:38:53.896649 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.896388    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5f787742-3eed-4ed6-b5cc-cecdca7fcd73" (UID: "5f787742-3eed-4ed6-b5cc-cecdca7fcd73"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 21:38:53.896649 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.896535    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5f787742-3eed-4ed6-b5cc-cecdca7fcd73" (UID: "5f787742-3eed-4ed6-b5cc-cecdca7fcd73"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:38:53.896649 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.896530    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5f787742-3eed-4ed6-b5cc-cecdca7fcd73" (UID: "5f787742-3eed-4ed6-b5cc-cecdca7fcd73"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:38:53.896649 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.896580    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istiod-ca-cert\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:38:53.896848 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.896724    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5f787742-3eed-4ed6-b5cc-cecdca7fcd73" (UID: "5f787742-3eed-4ed6-b5cc-cecdca7fcd73"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:38:53.896916 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.896895    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istio-data" (OuterVolumeSpecName: "istio-data") pod "5f787742-3eed-4ed6-b5cc-cecdca7fcd73" (UID: "5f787742-3eed-4ed6-b5cc-cecdca7fcd73"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:38:53.898257 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.898228    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5f787742-3eed-4ed6-b5cc-cecdca7fcd73" (UID: "5f787742-3eed-4ed6-b5cc-cecdca7fcd73"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 21:38:53.898401 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.898382    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istio-token" (OuterVolumeSpecName: "istio-token") pod "5f787742-3eed-4ed6-b5cc-cecdca7fcd73" (UID: "5f787742-3eed-4ed6-b5cc-cecdca7fcd73"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 21:38:53.898463 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.898409    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5f787742-3eed-4ed6-b5cc-cecdca7fcd73" (UID: "5f787742-3eed-4ed6-b5cc-cecdca7fcd73"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:38:53.898591 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.898570    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-kube-api-access-dl82z" (OuterVolumeSpecName: "kube-api-access-dl82z") pod "5f787742-3eed-4ed6-b5cc-cecdca7fcd73" (UID: "5f787742-3eed-4ed6-b5cc-cecdca7fcd73"). InnerVolumeSpecName "kube-api-access-dl82z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 21:38:53.997236 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.997208    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istio-data\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:38:53.997236 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.997233    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istio-podinfo\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:38:53.997236 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.997241    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istio-token\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:38:53.997433 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.997251    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-workload-socket\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:38:53.997433 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.997262    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-workload-certs\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:38:53.997433 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.997270    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-istio-envoy\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:38:53.997433 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.997277    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-credential-socket\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:38:53.997433 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:53.997285    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dl82z\" (UniqueName: \"kubernetes.io/projected/5f787742-3eed-4ed6-b5cc-cecdca7fcd73-kube-api-access-dl82z\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:38:54.346667 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:54.346627    2561 generic.go:356] "Generic (PLEG): container finished" podID="5f787742-3eed-4ed6-b5cc-cecdca7fcd73" containerID="deab21c8b3d4c50065ee324c6ddd67b30a063d57ab1da81414a69b8bac83469f" exitCode=0
Jul 20 21:38:54.346874 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:54.346699    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"
Jul 20 21:38:54.346874 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:54.346713    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd" event={"ID":"5f787742-3eed-4ed6-b5cc-cecdca7fcd73","Type":"ContainerDied","Data":"deab21c8b3d4c50065ee324c6ddd67b30a063d57ab1da81414a69b8bac83469f"}
Jul 20 21:38:54.346874 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:54.346791    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd" event={"ID":"5f787742-3eed-4ed6-b5cc-cecdca7fcd73","Type":"ContainerDied","Data":"d9b73e9e6b3009762ef0853eb73a52bedcf65a8c9d0dad4a2c5a793c0abfef0d"}
Jul 20 21:38:54.346874 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:54.346813    2561 scope.go:117] "RemoveContainer" containerID="deab21c8b3d4c50065ee324c6ddd67b30a063d57ab1da81414a69b8bac83469f"
Jul 20 21:38:54.356084 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:54.356057    2561 scope.go:117] "RemoveContainer" containerID="946f461493771a1bec21652dda40247a77ee3af59e589b3c247615365d9d3cf0"
Jul 20 21:38:54.366541 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:54.366519    2561 scope.go:117] "RemoveContainer" containerID="deab21c8b3d4c50065ee324c6ddd67b30a063d57ab1da81414a69b8bac83469f"
Jul 20 21:38:54.366806 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:38:54.366788    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"deab21c8b3d4c50065ee324c6ddd67b30a063d57ab1da81414a69b8bac83469f\": container with ID starting with deab21c8b3d4c50065ee324c6ddd67b30a063d57ab1da81414a69b8bac83469f not found: ID does not exist" containerID="deab21c8b3d4c50065ee324c6ddd67b30a063d57ab1da81414a69b8bac83469f"
Jul 20 21:38:54.366859 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:54.366814    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"deab21c8b3d4c50065ee324c6ddd67b30a063d57ab1da81414a69b8bac83469f"} err="failed to get container status \"deab21c8b3d4c50065ee324c6ddd67b30a063d57ab1da81414a69b8bac83469f\": rpc error: code = NotFound desc = could not find container \"deab21c8b3d4c50065ee324c6ddd67b30a063d57ab1da81414a69b8bac83469f\": container with ID starting with deab21c8b3d4c50065ee324c6ddd67b30a063d57ab1da81414a69b8bac83469f not found: ID does not exist"
Jul 20 21:38:54.366859 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:54.366830    2561 scope.go:117] "RemoveContainer" containerID="946f461493771a1bec21652dda40247a77ee3af59e589b3c247615365d9d3cf0"
Jul 20 21:38:54.367108 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:38:54.367091    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"946f461493771a1bec21652dda40247a77ee3af59e589b3c247615365d9d3cf0\": container with ID starting with 946f461493771a1bec21652dda40247a77ee3af59e589b3c247615365d9d3cf0 not found: ID does not exist" containerID="946f461493771a1bec21652dda40247a77ee3af59e589b3c247615365d9d3cf0"
Jul 20 21:38:54.367160 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:54.367115    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"946f461493771a1bec21652dda40247a77ee3af59e589b3c247615365d9d3cf0"} err="failed to get container status \"946f461493771a1bec21652dda40247a77ee3af59e589b3c247615365d9d3cf0\": rpc error: code = NotFound desc = could not find container \"946f461493771a1bec21652dda40247a77ee3af59e589b3c247615365d9d3cf0\": container with ID starting with 946f461493771a1bec21652dda40247a77ee3af59e589b3c247615365d9d3cf0 not found: ID does not exist"
Jul 20 21:38:54.371167 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:54.371146    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"]
Jul 20 21:38:54.377756 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:54.377735    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ff57b586-openshift-default-9bd644c-2gqmd"]
Jul 20 21:38:55.725596 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:38:55.725563    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f787742-3eed-4ed6-b5cc-cecdca7fcd73" path="/var/lib/kubelet/pods/5f787742-3eed-4ed6-b5cc-cecdca7fcd73/volumes"
Jul 20 21:39:08.904465 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:08.904423    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"]
Jul 20 21:39:08.904895 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:08.904773    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f787742-3eed-4ed6-b5cc-cecdca7fcd73" containerName="istio-proxy"
Jul 20 21:39:08.904895 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:08.904787    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f787742-3eed-4ed6-b5cc-cecdca7fcd73" containerName="istio-proxy"
Jul 20 21:39:08.904895 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:08.904851    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f787742-3eed-4ed6-b5cc-cecdca7fcd73" containerName="istio-proxy"
Jul 20 21:39:08.909130 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:08.909112    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:08.911869 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:08.911838    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-bf0666db-openshift-default-dockercfg-kh9rp\""
Jul 20 21:39:08.923885 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:08.923866    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"]
Jul 20 21:39:09.026946 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.026902    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l7xz\" (UniqueName: \"kubernetes.io/projected/34569a27-1267-42cd-87e8-9e5a84a58649-kube-api-access-7l7xz\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.027162 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.026950    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-istio-data\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.027162 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.026978    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-istio-envoy\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.027162 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.027047    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-workload-certs\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.027162 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.027092    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34569a27-1267-42cd-87e8-9e5a84a58649-istiod-ca-cert\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.027369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.027187    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-credential-socket\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.027369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.027235    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34569a27-1267-42cd-87e8-9e5a84a58649-istio-podinfo\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.027369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.027268    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34569a27-1267-42cd-87e8-9e5a84a58649-istio-token\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.027369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.027319    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-workload-socket\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.128500 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.128464    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34569a27-1267-42cd-87e8-9e5a84a58649-istio-podinfo\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.128677 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.128505    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34569a27-1267-42cd-87e8-9e5a84a58649-istio-token\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.128677 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.128541    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-workload-socket\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.128677 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.128615    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l7xz\" (UniqueName: \"kubernetes.io/projected/34569a27-1267-42cd-87e8-9e5a84a58649-kube-api-access-7l7xz\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.128677 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.128642    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-istio-data\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.128677 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.128670    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-istio-envoy\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.128937 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.128706    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-workload-certs\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.128937 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.128734    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34569a27-1267-42cd-87e8-9e5a84a58649-istiod-ca-cert\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.128937 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.128782    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-credential-socket\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.129126 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.129017    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-workload-socket\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.129126 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.129102    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-istio-data\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.129237 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.129202    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-workload-certs\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.129292 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.129249    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-credential-socket\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.129514 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.129492    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34569a27-1267-42cd-87e8-9e5a84a58649-istiod-ca-cert\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.131093 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.131045    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34569a27-1267-42cd-87e8-9e5a84a58649-istio-podinfo\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.131093 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.131086    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-istio-envoy\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.137232 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.137210    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34569a27-1267-42cd-87e8-9e5a84a58649-istio-token\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.137322 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.137241    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l7xz\" (UniqueName: \"kubernetes.io/projected/34569a27-1267-42cd-87e8-9e5a84a58649-kube-api-access-7l7xz\") pod \"e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") " pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.218730 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.218637    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:09.351806 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.351779    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"]
Jul 20 21:39:09.353887 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:39:09.353856    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34569a27_1267_42cd_87e8_9e5a84a58649.slice/crio-453c2d08733cbd8439721288ded0c3c51d0bf190d8d82e87fb05717f18c60ed8 WatchSource:0}: Error finding container 453c2d08733cbd8439721288ded0c3c51d0bf190d8d82e87fb05717f18c60ed8: Status 404 returned error can't find the container with id 453c2d08733cbd8439721288ded0c3c51d0bf190d8d82e87fb05717f18c60ed8
Jul 20 21:39:09.356091 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.356040    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:39:09.356155 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.356139    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:39:09.356195 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.356182    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:39:09.425431 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.425400    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq" event={"ID":"34569a27-1267-42cd-87e8-9e5a84a58649","Type":"ContainerStarted","Data":"0007f904c7f2de83ec32b3f554c4a345f24d914ec6004b1b952210771ae51cc7"}
Jul 20 21:39:09.425579 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.425450    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq" event={"ID":"34569a27-1267-42cd-87e8-9e5a84a58649","Type":"ContainerStarted","Data":"453c2d08733cbd8439721288ded0c3c51d0bf190d8d82e87fb05717f18c60ed8"}
Jul 20 21:39:09.447692 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:09.447640    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq" podStartSLOduration=1.447619399 podStartE2EDuration="1.447619399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:39:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:39:09.444922015 +0000 UTC m=+2190.348911591" watchObservedRunningTime="2026-07-20 21:39:09.447619399 +0000 UTC m=+2190.351608976"
Jul 20 21:39:10.219408 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:10.219373    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:10.224244 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:10.224222    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:10.430324 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:10.430296    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:10.431182 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:10.431163    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:18.493546 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:18.493511    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:39:26.354929 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:26.354783    2561 patch_prober.go:28] interesting pod/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": read tcp 10.133.0.2:39130->10.133.0.41:15021: read: connection reset by peer" start-of-body=
Jul 20 21:39:26.354929 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:26.354861    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq" podUID="34569a27-1267-42cd-87e8-9e5a84a58649" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": read tcp 10.133.0.2:39130->10.133.0.41:15021: read: connection reset by peer"
Jul 20 21:39:26.508004 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:26.507970    2561 generic.go:356] "Generic (PLEG): container finished" podID="34569a27-1267-42cd-87e8-9e5a84a58649" containerID="0007f904c7f2de83ec32b3f554c4a345f24d914ec6004b1b952210771ae51cc7" exitCode=0
Jul 20 21:39:26.508181 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:26.508042    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq" event={"ID":"34569a27-1267-42cd-87e8-9e5a84a58649","Type":"ContainerDied","Data":"0007f904c7f2de83ec32b3f554c4a345f24d914ec6004b1b952210771ae51cc7"}
Jul 20 21:39:26.508433 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:26.508418    2561 scope.go:117] "RemoveContainer" containerID="0007f904c7f2de83ec32b3f554c4a345f24d914ec6004b1b952210771ae51cc7"
Jul 20 21:39:26.509414 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:26.509385    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:39:26.509507 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:26.509450    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:39:26.509507 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:26.509481    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:39:27.514729 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:27.514691    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq" event={"ID":"34569a27-1267-42cd-87e8-9e5a84a58649","Type":"ContainerStarted","Data":"d8b3f5b3419df2ab4082497c5b31e062d03d4e2157e925c544a863a54d225b09"}
Jul 20 21:39:28.219047 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:28.219008    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:28.219322 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:28.219303    2561 patch_prober.go:28] interesting pod/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 20 21:39:28.219376 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:28.219358    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq" podUID="34569a27-1267-42cd-87e8-9e5a84a58649" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 20 21:39:29.219886 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:29.219846    2561 patch_prober.go:28] interesting pod/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 20 21:39:29.220391 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:29.219913    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq" podUID="34569a27-1267-42cd-87e8-9e5a84a58649" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 20 21:39:30.220064 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:30.220026    2561 patch_prober.go:28] interesting pod/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 20 21:39:30.220480 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:30.220101    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq" podUID="34569a27-1267-42cd-87e8-9e5a84a58649" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 20 21:39:31.219446 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:31.219408    2561 patch_prober.go:28] interesting pod/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 20 21:39:31.219636 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:31.219468    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq" podUID="34569a27-1267-42cd-87e8-9e5a84a58649" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 20 21:39:31.708102 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:31.708037    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:39:32.223618 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:32.223588    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:32.223980 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:32.223961    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:32.224532 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:32.224513    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:39:46.209416 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:46.209380    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:39:49.539290 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:49.539247    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf0666db-r84jd"]
Jul 20 21:39:49.539857 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:49.539764    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f787742-3eed-4ed6-b5cc-cecdca7fcd73" containerName="istio-proxy"
Jul 20 21:39:49.539857 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:49.539785    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f787742-3eed-4ed6-b5cc-cecdca7fcd73" containerName="istio-proxy"
Jul 20 21:39:49.539974 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:49.539874    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f787742-3eed-4ed6-b5cc-cecdca7fcd73" containerName="istio-proxy"
Jul 20 21:39:49.541773 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:49.541751    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf0666db-r84jd"
Jul 20 21:39:49.544125 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:49.544106    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 21:39:49.544248 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:49.544166    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qr6f9\""
Jul 20 21:39:49.553118 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:49.553097    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf0666db-r84jd"]
Jul 20 21:39:49.591602 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:49.591580    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e101efc-0e85-48ec-9ec9-528c6df1aa82-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-bf0666db-r84jd\" (UID: \"3e101efc-0e85-48ec-9ec9-528c6df1aa82\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf0666db-r84jd"
Jul 20 21:39:49.692774 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:49.692743    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e101efc-0e85-48ec-9ec9-528c6df1aa82-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-bf0666db-r84jd\" (UID: \"3e101efc-0e85-48ec-9ec9-528c6df1aa82\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf0666db-r84jd"
Jul 20 21:39:49.693375 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:49.693356    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e101efc-0e85-48ec-9ec9-528c6df1aa82-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-bf0666db-r84jd\" (UID: \"3e101efc-0e85-48ec-9ec9-528c6df1aa82\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf0666db-r84jd"
Jul 20 21:39:49.852057 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:49.851986    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf0666db-r84jd"
Jul 20 21:39:49.985022 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:49.984997    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf0666db-r84jd"]
Jul 20 21:39:49.986804 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:39:49.986775    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e101efc_0e85_48ec_9ec9_528c6df1aa82.slice/crio-100da5fbc8654758429a148fb639e3ae3687e15d91abc3893266fc14f8b349bd WatchSource:0}: Error finding container 100da5fbc8654758429a148fb639e3ae3687e15d91abc3893266fc14f8b349bd: Status 404 returned error can't find the container with id 100da5fbc8654758429a148fb639e3ae3687e15d91abc3893266fc14f8b349bd
Jul 20 21:39:50.629348 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:50.629315    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf0666db-r84jd" event={"ID":"3e101efc-0e85-48ec-9ec9-528c6df1aa82","Type":"ContainerStarted","Data":"ba9676958210a11e4ed76dbb42d8151e444eeade9acbcf7a560168c302eab967"}
Jul 20 21:39:50.629710 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:50.629366    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf0666db-r84jd" event={"ID":"3e101efc-0e85-48ec-9ec9-528c6df1aa82","Type":"ContainerStarted","Data":"100da5fbc8654758429a148fb639e3ae3687e15d91abc3893266fc14f8b349bd"}
Jul 20 21:39:50.648034 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:50.647991    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf0666db-r84jd" podStartSLOduration=1.6479763630000002 podStartE2EDuration="1.647976363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:39:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:39:50.646416163 +0000 UTC m=+2231.550405740" watchObservedRunningTime="2026-07-20 21:39:50.647976363 +0000 UTC m=+2231.551965957"
Jul 20 21:39:51.635440 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:51.635406    2561 generic.go:356] "Generic (PLEG): container finished" podID="3e101efc-0e85-48ec-9ec9-528c6df1aa82" containerID="ba9676958210a11e4ed76dbb42d8151e444eeade9acbcf7a560168c302eab967" exitCode=0
Jul 20 21:39:51.635884 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:51.635450    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf0666db-r84jd" event={"ID":"3e101efc-0e85-48ec-9ec9-528c6df1aa82","Type":"ContainerDied","Data":"ba9676958210a11e4ed76dbb42d8151e444eeade9acbcf7a560168c302eab967"}
Jul 20 21:39:52.764104 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:52.764058    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf0666db-r84jd"
Jul 20 21:39:52.820470 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:52.820437    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e101efc-0e85-48ec-9ec9-528c6df1aa82-maas-api-service-ca\") pod \"3e101efc-0e85-48ec-9ec9-528c6df1aa82\" (UID: \"3e101efc-0e85-48ec-9ec9-528c6df1aa82\") "
Jul 20 21:39:52.820782 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:52.820761    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e101efc-0e85-48ec-9ec9-528c6df1aa82-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3e101efc-0e85-48ec-9ec9-528c6df1aa82" (UID: "3e101efc-0e85-48ec-9ec9-528c6df1aa82"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 21:39:52.921372 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:52.921292    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e101efc-0e85-48ec-9ec9-528c6df1aa82-maas-api-service-ca\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:39:53.646238 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:53.646206    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf0666db-r84jd" event={"ID":"3e101efc-0e85-48ec-9ec9-528c6df1aa82","Type":"ContainerDied","Data":"100da5fbc8654758429a148fb639e3ae3687e15d91abc3893266fc14f8b349bd"}
Jul 20 21:39:53.646432 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:53.646269    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="100da5fbc8654758429a148fb639e3ae3687e15d91abc3893266fc14f8b349bd"
Jul 20 21:39:53.646432 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:39:53.646212    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf0666db-r84jd"
Jul 20 21:40:03.598691 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:40:03.598608    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:40:09.621208 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:40:09.621175    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf0666db-r84jd"]
Jul 20 21:40:09.625780 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:40:09.625756    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bf0666db-r84jd"]
Jul 20 21:40:09.725247 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:40:09.725216    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e101efc-0e85-48ec-9ec9-528c6df1aa82" path="/var/lib/kubelet/pods/3e101efc-0e85-48ec-9ec9-528c6df1aa82/volumes"
Jul 20 21:40:14.097552 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:40:14.097518    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:40:39.817189 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:40:39.817157    2561 scope.go:117] "RemoveContainer" containerID="cf33d3948aab3532e43e30044577d2f7f0eb210b4f59e2b015ba8ea5c37968c8"
Jul 20 21:40:39.825205 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:40:39.825184    2561 scope.go:117] "RemoveContainer" containerID="1faf8ecf3b7559d019308d747dd345beafc1933783ebcc5da107dace05af1767"
Jul 20 21:41:00.580737 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:00.580705    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"]
Jul 20 21:41:00.581151 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:00.580915    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq" podUID="34569a27-1267-42cd-87e8-9e5a84a58649" containerName="istio-proxy" containerID="cri-o://d8b3f5b3419df2ab4082497c5b31e062d03d4e2157e925c544a863a54d225b09" gracePeriod=30
Jul 20 21:41:02.226521 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:02.226488    2561 patch_prober.go:28] interesting pod/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 20 21:41:02.226879 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:02.226552    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq" podUID="34569a27-1267-42cd-87e8-9e5a84a58649" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 20 21:41:05.824962 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.824940    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:41:05.947885 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.947795    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-credential-socket\") pod \"34569a27-1267-42cd-87e8-9e5a84a58649\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") "
Jul 20 21:41:05.947885 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.947839    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34569a27-1267-42cd-87e8-9e5a84a58649-istio-podinfo\") pod \"34569a27-1267-42cd-87e8-9e5a84a58649\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") "
Jul 20 21:41:05.948141 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.947895    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-istio-data\") pod \"34569a27-1267-42cd-87e8-9e5a84a58649\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") "
Jul 20 21:41:05.948141 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.947913    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-workload-certs\") pod \"34569a27-1267-42cd-87e8-9e5a84a58649\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") "
Jul 20 21:41:05.948141 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.947930    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-workload-socket\") pod \"34569a27-1267-42cd-87e8-9e5a84a58649\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") "
Jul 20 21:41:05.948141 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.947956    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34569a27-1267-42cd-87e8-9e5a84a58649-istiod-ca-cert\") pod \"34569a27-1267-42cd-87e8-9e5a84a58649\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") "
Jul 20 21:41:05.948141 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.947982    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34569a27-1267-42cd-87e8-9e5a84a58649-istio-token\") pod \"34569a27-1267-42cd-87e8-9e5a84a58649\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") "
Jul 20 21:41:05.948141 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.948067    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-istio-envoy\") pod \"34569a27-1267-42cd-87e8-9e5a84a58649\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") "
Jul 20 21:41:05.948141 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.948142    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7l7xz\" (UniqueName: \"kubernetes.io/projected/34569a27-1267-42cd-87e8-9e5a84a58649-kube-api-access-7l7xz\") pod \"34569a27-1267-42cd-87e8-9e5a84a58649\" (UID: \"34569a27-1267-42cd-87e8-9e5a84a58649\") "
Jul 20 21:41:05.948502 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.948188    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "34569a27-1267-42cd-87e8-9e5a84a58649" (UID: "34569a27-1267-42cd-87e8-9e5a84a58649"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:41:05.948502 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.948263    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "34569a27-1267-42cd-87e8-9e5a84a58649" (UID: "34569a27-1267-42cd-87e8-9e5a84a58649"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:41:05.948502 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.948281    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "34569a27-1267-42cd-87e8-9e5a84a58649" (UID: "34569a27-1267-42cd-87e8-9e5a84a58649"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:41:05.948502 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.948395    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34569a27-1267-42cd-87e8-9e5a84a58649-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "34569a27-1267-42cd-87e8-9e5a84a58649" (UID: "34569a27-1267-42cd-87e8-9e5a84a58649"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 21:41:05.948502 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.948437    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-credential-socket\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:41:05.948502 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.948456    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-workload-certs\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:41:05.948502 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.948468    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-workload-socket\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:41:05.948869 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.948735    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-istio-data" (OuterVolumeSpecName: "istio-data") pod "34569a27-1267-42cd-87e8-9e5a84a58649" (UID: "34569a27-1267-42cd-87e8-9e5a84a58649"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:41:05.950318 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.950289    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34569a27-1267-42cd-87e8-9e5a84a58649-kube-api-access-7l7xz" (OuterVolumeSpecName: "kube-api-access-7l7xz") pod "34569a27-1267-42cd-87e8-9e5a84a58649" (UID: "34569a27-1267-42cd-87e8-9e5a84a58649"). InnerVolumeSpecName "kube-api-access-7l7xz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 21:41:05.950318 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.950296    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/34569a27-1267-42cd-87e8-9e5a84a58649-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "34569a27-1267-42cd-87e8-9e5a84a58649" (UID: "34569a27-1267-42cd-87e8-9e5a84a58649"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 21:41:05.950318 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.950316    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34569a27-1267-42cd-87e8-9e5a84a58649-istio-token" (OuterVolumeSpecName: "istio-token") pod "34569a27-1267-42cd-87e8-9e5a84a58649" (UID: "34569a27-1267-42cd-87e8-9e5a84a58649"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 21:41:05.950498 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.950433    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "34569a27-1267-42cd-87e8-9e5a84a58649" (UID: "34569a27-1267-42cd-87e8-9e5a84a58649"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:41:05.986949 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.986919    2561 generic.go:356] "Generic (PLEG): container finished" podID="34569a27-1267-42cd-87e8-9e5a84a58649" containerID="d8b3f5b3419df2ab4082497c5b31e062d03d4e2157e925c544a863a54d225b09" exitCode=0
Jul 20 21:41:05.987061 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.986965    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"
Jul 20 21:41:05.987061 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.987000    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq" event={"ID":"34569a27-1267-42cd-87e8-9e5a84a58649","Type":"ContainerDied","Data":"d8b3f5b3419df2ab4082497c5b31e062d03d4e2157e925c544a863a54d225b09"}
Jul 20 21:41:05.987061 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.987050    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq" event={"ID":"34569a27-1267-42cd-87e8-9e5a84a58649","Type":"ContainerDied","Data":"453c2d08733cbd8439721288ded0c3c51d0bf190d8d82e87fb05717f18c60ed8"}
Jul 20 21:41:05.987187 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.987087    2561 scope.go:117] "RemoveContainer" containerID="d8b3f5b3419df2ab4082497c5b31e062d03d4e2157e925c544a863a54d225b09"
Jul 20 21:41:05.995790 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:05.995775    2561 scope.go:117] "RemoveContainer" containerID="0007f904c7f2de83ec32b3f554c4a345f24d914ec6004b1b952210771ae51cc7"
Jul 20 21:41:06.003181 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:06.003166    2561 scope.go:117] "RemoveContainer" containerID="d8b3f5b3419df2ab4082497c5b31e062d03d4e2157e925c544a863a54d225b09"
Jul 20 21:41:06.003429 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:41:06.003411    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8b3f5b3419df2ab4082497c5b31e062d03d4e2157e925c544a863a54d225b09\": container with ID starting with d8b3f5b3419df2ab4082497c5b31e062d03d4e2157e925c544a863a54d225b09 not found: ID does not exist" containerID="d8b3f5b3419df2ab4082497c5b31e062d03d4e2157e925c544a863a54d225b09"
Jul 20 21:41:06.003466 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:06.003437    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8b3f5b3419df2ab4082497c5b31e062d03d4e2157e925c544a863a54d225b09"} err="failed to get container status \"d8b3f5b3419df2ab4082497c5b31e062d03d4e2157e925c544a863a54d225b09\": rpc error: code = NotFound desc = could not find container \"d8b3f5b3419df2ab4082497c5b31e062d03d4e2157e925c544a863a54d225b09\": container with ID starting with d8b3f5b3419df2ab4082497c5b31e062d03d4e2157e925c544a863a54d225b09 not found: ID does not exist"
Jul 20 21:41:06.003466 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:06.003453    2561 scope.go:117] "RemoveContainer" containerID="0007f904c7f2de83ec32b3f554c4a345f24d914ec6004b1b952210771ae51cc7"
Jul 20 21:41:06.003652 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:41:06.003634    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0007f904c7f2de83ec32b3f554c4a345f24d914ec6004b1b952210771ae51cc7\": container with ID starting with 0007f904c7f2de83ec32b3f554c4a345f24d914ec6004b1b952210771ae51cc7 not found: ID does not exist" containerID="0007f904c7f2de83ec32b3f554c4a345f24d914ec6004b1b952210771ae51cc7"
Jul 20 21:41:06.003687 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:06.003660    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0007f904c7f2de83ec32b3f554c4a345f24d914ec6004b1b952210771ae51cc7"} err="failed to get container status \"0007f904c7f2de83ec32b3f554c4a345f24d914ec6004b1b952210771ae51cc7\": rpc error: code = NotFound desc = could not find container \"0007f904c7f2de83ec32b3f554c4a345f24d914ec6004b1b952210771ae51cc7\": container with ID starting with 0007f904c7f2de83ec32b3f554c4a345f24d914ec6004b1b952210771ae51cc7 not found: ID does not exist"
Jul 20 21:41:06.011605 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:06.011583    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"]
Jul 20 21:41:06.017440 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:06.017419    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-bf0666db-openshift-default-677f4fbcff-9xmpq"]
Jul 20 21:41:06.049569 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:06.049545    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-istio-envoy\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:41:06.049569 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:06.049569    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7l7xz\" (UniqueName: \"kubernetes.io/projected/34569a27-1267-42cd-87e8-9e5a84a58649-kube-api-access-7l7xz\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:41:06.049673 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:06.049580    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34569a27-1267-42cd-87e8-9e5a84a58649-istio-podinfo\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:41:06.049673 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:06.049589    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34569a27-1267-42cd-87e8-9e5a84a58649-istio-data\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:41:06.049673 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:06.049597    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34569a27-1267-42cd-87e8-9e5a84a58649-istiod-ca-cert\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:41:06.049673 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:06.049605    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34569a27-1267-42cd-87e8-9e5a84a58649-istio-token\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:41:07.726420 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:07.726386    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34569a27-1267-42cd-87e8-9e5a84a58649" path="/var/lib/kubelet/pods/34569a27-1267-42cd-87e8-9e5a84a58649/volumes"
Jul 20 21:41:11.087157 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:11.087119    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:41:19.097174 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:19.097137    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:41:39.849904 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:39.849840    2561 scope.go:117] "RemoveContainer" containerID="f493ccc8cdc6ad9e0f314f09c717e1efe4a90e0fa3f00c5397b316175628c22d"
Jul 20 21:41:57.385688 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.385658    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"]
Jul 20 21:41:57.386175 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.385978    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34569a27-1267-42cd-87e8-9e5a84a58649" containerName="istio-proxy"
Jul 20 21:41:57.386175 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.385996    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="34569a27-1267-42cd-87e8-9e5a84a58649" containerName="istio-proxy"
Jul 20 21:41:57.386175 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.386007    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34569a27-1267-42cd-87e8-9e5a84a58649" containerName="istio-proxy"
Jul 20 21:41:57.386175 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.386012    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="34569a27-1267-42cd-87e8-9e5a84a58649" containerName="istio-proxy"
Jul 20 21:41:57.386175 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.386028    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e101efc-0e85-48ec-9ec9-528c6df1aa82" containerName="revoke-keys"
Jul 20 21:41:57.386175 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.386034    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e101efc-0e85-48ec-9ec9-528c6df1aa82" containerName="revoke-keys"
Jul 20 21:41:57.386175 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.386119    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="34569a27-1267-42cd-87e8-9e5a84a58649" containerName="istio-proxy"
Jul 20 21:41:57.386175 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.386127    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e101efc-0e85-48ec-9ec9-528c6df1aa82" containerName="revoke-keys"
Jul 20 21:41:57.386175 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.386135    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="34569a27-1267-42cd-87e8-9e5a84a58649" containerName="istio-proxy"
Jul 20 21:41:57.388922 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.388905    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.391425 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.391404    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-b7f42a-openshift-default-dockercfg-59k4x\""
Jul 20 21:41:57.401334 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.401309    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfhpb\" (UniqueName: \"kubernetes.io/projected/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-kube-api-access-qfhpb\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.401595 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.401353    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istio-data\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.401595 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.401399    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-credential-socket\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.401595 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.401422    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istio-envoy\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.401595 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.401464    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-workload-socket\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.401595 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.401499    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istio-podinfo\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.401595 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.401528    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istio-token\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.401935 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.401619    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-workload-certs\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.401935 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.401651    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istiod-ca-cert\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.403310 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.403281    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"]
Jul 20 21:41:57.502026 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.501992    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-workload-socket\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.502026 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.502030    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istio-podinfo\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.502312 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.502052    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istio-token\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.502312 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.502210    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-workload-certs\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.502312 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.502262    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istiod-ca-cert\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.502486 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.502333    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfhpb\" (UniqueName: \"kubernetes.io/projected/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-kube-api-access-qfhpb\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.502486 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.502382    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istio-data\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.502486 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.502436    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-workload-socket\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.502604 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.502491    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-workload-certs\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.502714 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.502694    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-credential-socket\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.502761 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.502719    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istio-data\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.502761 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.502726    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istio-envoy\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.502997 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.502976    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istiod-ca-cert\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.503069 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.503033    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-credential-socket\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.504610 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.504594    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istio-podinfo\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.504792 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.504775    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istio-envoy\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.510900 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.510883    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istio-token\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.511424 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.511393    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfhpb\" (UniqueName: \"kubernetes.io/projected/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-kube-api-access-qfhpb\") pod \"e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") " pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.702281 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.702199    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:57.844128 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.844102    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"]
Jul 20 21:41:57.846309 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:41:57.846277    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95e61aaa_df4d_4d13_bb33_06d0e2aef4c3.slice/crio-b3f3cbc52b4a700ec1f42ec5b6ae258aa42e0f7dfae6b7d87927f38316869233 WatchSource:0}: Error finding container b3f3cbc52b4a700ec1f42ec5b6ae258aa42e0f7dfae6b7d87927f38316869233: Status 404 returned error can't find the container with id b3f3cbc52b4a700ec1f42ec5b6ae258aa42e0f7dfae6b7d87927f38316869233
Jul 20 21:41:57.848794 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.848762    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:41:57.848861 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.848826    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:41:57.848861 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:57.848854    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:41:58.228611 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:58.228572    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm" event={"ID":"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3","Type":"ContainerStarted","Data":"fb42ac3fc56a530c06cdac5bd5022760af7a1e3e326af786a0f9d06fc7c6e458"}
Jul 20 21:41:58.228785 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:58.228641    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm" event={"ID":"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3","Type":"ContainerStarted","Data":"b3f3cbc52b4a700ec1f42ec5b6ae258aa42e0f7dfae6b7d87927f38316869233"}
Jul 20 21:41:58.251816 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:58.251750    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm" podStartSLOduration=1.251734791 podStartE2EDuration="1.251734791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:41:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:41:58.248977058 +0000 UTC m=+2359.152966635" watchObservedRunningTime="2026-07-20 21:41:58.251734791 +0000 UTC m=+2359.155724369"
Jul 20 21:41:58.703119 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:58.703063    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:58.707810 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:58.707787    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:59.233784 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:59.233753    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:41:59.234650 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:41:59.234629    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:42:06.108827 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:06.108791    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:42:14.235771 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:14.235735    2561 patch_prober.go:28] interesting pod/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Jul 20 21:42:14.236220 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:14.235806    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm" podUID="95e61aaa-df4d-4d13-bb33-06d0e2aef4c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Jul 20 21:42:14.310367 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:14.310282    2561 generic.go:356] "Generic (PLEG): container finished" podID="95e61aaa-df4d-4d13-bb33-06d0e2aef4c3" containerID="fb42ac3fc56a530c06cdac5bd5022760af7a1e3e326af786a0f9d06fc7c6e458" exitCode=0
Jul 20 21:42:14.310522 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:14.310358    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm" event={"ID":"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3","Type":"ContainerDied","Data":"fb42ac3fc56a530c06cdac5bd5022760af7a1e3e326af786a0f9d06fc7c6e458"}
Jul 20 21:42:14.310775 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:14.310759    2561 scope.go:117] "RemoveContainer" containerID="fb42ac3fc56a530c06cdac5bd5022760af7a1e3e326af786a0f9d06fc7c6e458"
Jul 20 21:42:14.311767 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:14.311740    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:42:14.311855 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:14.311802    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:42:14.311855 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:14.311833    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:42:15.316937 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:15.316904    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm" event={"ID":"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3","Type":"ContainerStarted","Data":"db53f07590e4372c55ea3fbd755eb6799b0345670ca35eb02b269b1311706b9f"}
Jul 20 21:42:15.702698 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:15.702595    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:42:15.702876 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:15.702834    2561 patch_prober.go:28] interesting pod/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Jul 20 21:42:15.702956 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:15.702888    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm" podUID="95e61aaa-df4d-4d13-bb33-06d0e2aef4c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Jul 20 21:42:16.703120 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:16.703055    2561 patch_prober.go:28] interesting pod/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Jul 20 21:42:16.703546 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:16.703139    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm" podUID="95e61aaa-df4d-4d13-bb33-06d0e2aef4c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Jul 20 21:42:17.703402 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:17.703356    2561 patch_prober.go:28] interesting pod/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Jul 20 21:42:17.703818 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:17.703433    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm" podUID="95e61aaa-df4d-4d13-bb33-06d0e2aef4c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Jul 20 21:42:18.702959 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:18.702918    2561 patch_prober.go:28] interesting pod/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Jul 20 21:42:18.703158 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:18.702991    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm" podUID="95e61aaa-df4d-4d13-bb33-06d0e2aef4c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Jul 20 21:42:19.706955 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:19.706920    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:42:19.707504 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:19.707328    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:42:19.708105 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:19.708088    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:42:20.485134 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:20.485091    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:42:38.287369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:38.287334    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:42:39.871406 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:39.871373    2561 scope.go:117] "RemoveContainer" containerID="39206cec519b6de63697f7842853242286f5c8b921b505e2e13cea15112f5386"
Jul 20 21:42:40.513164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:40.513130    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/1.log"
Jul 20 21:42:40.519659 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:42:40.519633    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/1.log"
Jul 20 21:43:14.895142 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:43:14.895104    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:43:54.669005 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:43:54.668969    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b7f42a-jcs9x"]
Jul 20 21:43:54.677033 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:43:54.677007    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b7f42a-jcs9x"
Jul 20 21:43:54.679520 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:43:54.679495    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qr6f9\""
Jul 20 21:43:54.679653 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:43:54.679524    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 21:43:54.680873 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:43:54.680852    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b7f42a-jcs9x"]
Jul 20 21:43:54.815667 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:43:54.815636    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0505c5a-df08-488d-9628-226f2beff996-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-b7f42a-jcs9x\" (UID: \"c0505c5a-df08-488d-9628-226f2beff996\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b7f42a-jcs9x"
Jul 20 21:43:54.916756 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:43:54.916727    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0505c5a-df08-488d-9628-226f2beff996-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-b7f42a-jcs9x\" (UID: \"c0505c5a-df08-488d-9628-226f2beff996\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b7f42a-jcs9x"
Jul 20 21:43:54.917372 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:43:54.917352    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0505c5a-df08-488d-9628-226f2beff996-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-b7f42a-jcs9x\" (UID: \"c0505c5a-df08-488d-9628-226f2beff996\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b7f42a-jcs9x"
Jul 20 21:43:54.989324 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:43:54.989260    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b7f42a-jcs9x"
Jul 20 21:43:55.123377 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:43:55.123352    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b7f42a-jcs9x"]
Jul 20 21:43:55.125021 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:43:55.124997    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0505c5a_df08_488d_9628_226f2beff996.slice/crio-8a8c86074c1cae216d6039e48c541501464e313761edab6556c6b86e9502d0fe WatchSource:0}: Error finding container 8a8c86074c1cae216d6039e48c541501464e313761edab6556c6b86e9502d0fe: Status 404 returned error can't find the container with id 8a8c86074c1cae216d6039e48c541501464e313761edab6556c6b86e9502d0fe
Jul 20 21:43:55.782013 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:43:55.781918    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b7f42a-jcs9x" event={"ID":"c0505c5a-df08-488d-9628-226f2beff996","Type":"ContainerStarted","Data":"9c9878c72710c5df0cbf5b8610a8eb9c7007ae2c276ca66ac85e768794e4166d"}
Jul 20 21:43:55.782013 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:43:55.781972    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b7f42a-jcs9x" event={"ID":"c0505c5a-df08-488d-9628-226f2beff996","Type":"ContainerStarted","Data":"8a8c86074c1cae216d6039e48c541501464e313761edab6556c6b86e9502d0fe"}
Jul 20 21:43:55.802223 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:43:55.802178    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b7f42a-jcs9x" podStartSLOduration=1.802162764 podStartE2EDuration="1.802162764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:43:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:43:55.800598632 +0000 UTC m=+2476.704588209" watchObservedRunningTime="2026-07-20 21:43:55.802162764 +0000 UTC m=+2476.706152341"
Jul 20 21:43:56.788454 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:43:56.788420    2561 generic.go:356] "Generic (PLEG): container finished" podID="c0505c5a-df08-488d-9628-226f2beff996" containerID="9c9878c72710c5df0cbf5b8610a8eb9c7007ae2c276ca66ac85e768794e4166d" exitCode=0
Jul 20 21:43:56.788454 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:43:56.788459    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b7f42a-jcs9x" event={"ID":"c0505c5a-df08-488d-9628-226f2beff996","Type":"ContainerDied","Data":"9c9878c72710c5df0cbf5b8610a8eb9c7007ae2c276ca66ac85e768794e4166d"}
Jul 20 21:43:57.920267 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:43:57.920240    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b7f42a-jcs9x"
Jul 20 21:43:58.042808 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:43:58.042780    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0505c5a-df08-488d-9628-226f2beff996-maas-api-service-ca\") pod \"c0505c5a-df08-488d-9628-226f2beff996\" (UID: \"c0505c5a-df08-488d-9628-226f2beff996\") "
Jul 20 21:43:58.043185 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:43:58.043164    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0505c5a-df08-488d-9628-226f2beff996-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c0505c5a-df08-488d-9628-226f2beff996" (UID: "c0505c5a-df08-488d-9628-226f2beff996"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 21:43:58.144171 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:43:58.144092    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0505c5a-df08-488d-9628-226f2beff996-maas-api-service-ca\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:43:58.799240 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:43:58.799207    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b7f42a-jcs9x"
Jul 20 21:43:58.799393 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:43:58.799213    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b7f42a-jcs9x" event={"ID":"c0505c5a-df08-488d-9628-226f2beff996","Type":"ContainerDied","Data":"8a8c86074c1cae216d6039e48c541501464e313761edab6556c6b86e9502d0fe"}
Jul 20 21:43:58.799393 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:43:58.799333    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a8c86074c1cae216d6039e48c541501464e313761edab6556c6b86e9502d0fe"
Jul 20 21:44:01.082401 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:01.082367    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:44:09.761694 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:09.761658    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b7f42a-jcs9x"]
Jul 20 21:44:09.764669 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:09.764643    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b7f42a-jcs9x"]
Jul 20 21:44:11.082777 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:11.082742    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:44:11.725875 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:11.725839    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0505c5a-df08-488d-9628-226f2beff996" path="/var/lib/kubelet/pods/c0505c5a-df08-488d-9628-226f2beff996/volumes"
Jul 20 21:44:16.692117 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:16.692059    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"]
Jul 20 21:44:16.692579 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:16.692292    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm" podUID="95e61aaa-df4d-4d13-bb33-06d0e2aef4c3" containerName="istio-proxy" containerID="cri-o://db53f07590e4372c55ea3fbd755eb6799b0345670ca35eb02b269b1311706b9f" gracePeriod=30
Jul 20 21:44:19.709953 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:19.709913    2561 patch_prober.go:28] interesting pod/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 20 21:44:19.710345 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:19.709990    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm" podUID="95e61aaa-df4d-4d13-bb33-06d0e2aef4c3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 20 21:44:21.913764 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:21.913730    2561 generic.go:356] "Generic (PLEG): container finished" podID="95e61aaa-df4d-4d13-bb33-06d0e2aef4c3" containerID="db53f07590e4372c55ea3fbd755eb6799b0345670ca35eb02b269b1311706b9f" exitCode=0
Jul 20 21:44:21.914164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:21.913773    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm" event={"ID":"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3","Type":"ContainerDied","Data":"db53f07590e4372c55ea3fbd755eb6799b0345670ca35eb02b269b1311706b9f"}
Jul 20 21:44:21.914164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:21.913828    2561 scope.go:117] "RemoveContainer" containerID="fb42ac3fc56a530c06cdac5bd5022760af7a1e3e326af786a0f9d06fc7c6e458"
Jul 20 21:44:21.936560 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:21.936541    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:44:21.958488 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:21.958426    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istio-podinfo\") pod \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") "
Jul 20 21:44:21.958488 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:21.958482    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istio-envoy\") pod \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") "
Jul 20 21:44:21.958636 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:21.958499    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-workload-socket\") pod \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") "
Jul 20 21:44:21.958636 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:21.958516    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istio-token\") pod \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") "
Jul 20 21:44:21.958636 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:21.958558    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-credential-socket\") pod \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") "
Jul 20 21:44:21.958636 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:21.958590    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istiod-ca-cert\") pod \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") "
Jul 20 21:44:21.958636 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:21.958618    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-workload-certs\") pod \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") "
Jul 20 21:44:21.958889 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:21.958644    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istio-data\") pod \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") "
Jul 20 21:44:21.958889 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:21.958671    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qfhpb\" (UniqueName: \"kubernetes.io/projected/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-kube-api-access-qfhpb\") pod \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\" (UID: \"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3\") "
Jul 20 21:44:21.958889 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:21.958780    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "95e61aaa-df4d-4d13-bb33-06d0e2aef4c3" (UID: "95e61aaa-df4d-4d13-bb33-06d0e2aef4c3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:44:21.959052 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:21.958974    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-workload-socket\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:44:21.959126 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:21.959052    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "95e61aaa-df4d-4d13-bb33-06d0e2aef4c3" (UID: "95e61aaa-df4d-4d13-bb33-06d0e2aef4c3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 21:44:21.959126 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:21.959051    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "95e61aaa-df4d-4d13-bb33-06d0e2aef4c3" (UID: "95e61aaa-df4d-4d13-bb33-06d0e2aef4c3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:44:21.959239 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:21.959161    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "95e61aaa-df4d-4d13-bb33-06d0e2aef4c3" (UID: "95e61aaa-df4d-4d13-bb33-06d0e2aef4c3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:44:21.960416 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:21.960373    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istio-data" (OuterVolumeSpecName: "istio-data") pod "95e61aaa-df4d-4d13-bb33-06d0e2aef4c3" (UID: "95e61aaa-df4d-4d13-bb33-06d0e2aef4c3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:44:21.961547 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:21.961329    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-kube-api-access-qfhpb" (OuterVolumeSpecName: "kube-api-access-qfhpb") pod "95e61aaa-df4d-4d13-bb33-06d0e2aef4c3" (UID: "95e61aaa-df4d-4d13-bb33-06d0e2aef4c3"). InnerVolumeSpecName "kube-api-access-qfhpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 21:44:21.962156 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:21.962118    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "95e61aaa-df4d-4d13-bb33-06d0e2aef4c3" (UID: "95e61aaa-df4d-4d13-bb33-06d0e2aef4c3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 21:44:21.962156 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:21.962129    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "95e61aaa-df4d-4d13-bb33-06d0e2aef4c3" (UID: "95e61aaa-df4d-4d13-bb33-06d0e2aef4c3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:44:21.962367 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:21.962191    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istio-token" (OuterVolumeSpecName: "istio-token") pod "95e61aaa-df4d-4d13-bb33-06d0e2aef4c3" (UID: "95e61aaa-df4d-4d13-bb33-06d0e2aef4c3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 21:44:22.060236 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:22.060199    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istio-podinfo\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:44:22.060236 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:22.060228    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istio-envoy\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:44:22.060236 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:22.060237    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istio-token\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:44:22.060236 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:22.060245    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-credential-socket\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:44:22.060491 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:22.060254    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istiod-ca-cert\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:44:22.060491 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:22.060262    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-workload-certs\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:44:22.060491 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:22.060270    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-istio-data\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:44:22.060491 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:22.060278    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfhpb\" (UniqueName: \"kubernetes.io/projected/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3-kube-api-access-qfhpb\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:44:22.920788 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:22.920756    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"
Jul 20 21:44:22.921234 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:22.920763    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm" event={"ID":"95e61aaa-df4d-4d13-bb33-06d0e2aef4c3","Type":"ContainerDied","Data":"b3f3cbc52b4a700ec1f42ec5b6ae258aa42e0f7dfae6b7d87927f38316869233"}
Jul 20 21:44:22.921234 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:22.920886    2561 scope.go:117] "RemoveContainer" containerID="db53f07590e4372c55ea3fbd755eb6799b0345670ca35eb02b269b1311706b9f"
Jul 20 21:44:22.956708 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:22.956680    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"]
Jul 20 21:44:22.960369 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:22.960347    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-b7f42a-openshift-default-ccbddb88f-d62pm"]
Jul 20 21:44:23.726233 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:23.726203    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95e61aaa-df4d-4d13-bb33-06d0e2aef4c3" path="/var/lib/kubelet/pods/95e61aaa-df4d-4d13-bb33-06d0e2aef4c3/volumes"
Jul 20 21:44:30.688611 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:30.688578    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:44:49.007749 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.007713    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"]
Jul 20 21:44:49.008215 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.008060    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95e61aaa-df4d-4d13-bb33-06d0e2aef4c3" containerName="istio-proxy"
Jul 20 21:44:49.008215 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.008099    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="95e61aaa-df4d-4d13-bb33-06d0e2aef4c3" containerName="istio-proxy"
Jul 20 21:44:49.008215 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.008119    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0505c5a-df08-488d-9628-226f2beff996" containerName="revoke-keys"
Jul 20 21:44:49.008215 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.008124    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0505c5a-df08-488d-9628-226f2beff996" containerName="revoke-keys"
Jul 20 21:44:49.008215 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.008182    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0505c5a-df08-488d-9628-226f2beff996" containerName="revoke-keys"
Jul 20 21:44:49.008215 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.008190    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="95e61aaa-df4d-4d13-bb33-06d0e2aef4c3" containerName="istio-proxy"
Jul 20 21:44:49.008215 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.008198    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="95e61aaa-df4d-4d13-bb33-06d0e2aef4c3" containerName="istio-proxy"
Jul 20 21:44:49.011012 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.010992    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.014120 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.014101    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-08ebd1-openshift-default-dockercfg-sqml7\""
Jul 20 21:44:49.036886 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.036863    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"]
Jul 20 21:44:49.095537 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.095502    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-workload-socket\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.095755 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.095562    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istio-envoy\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.095755 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.095588    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istio-podinfo\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.095755 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.095618    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt9f8\" (UniqueName: \"kubernetes.io/projected/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-kube-api-access-vt9f8\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.095755 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.095690    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-credential-socket\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.095755 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.095727    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istio-token\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.096016 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.095812    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istio-data\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.096016 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.095837    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istiod-ca-cert\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.096016 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.095882    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-workload-certs\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.196501 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.196468    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istio-envoy\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.196501 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.196505    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istio-podinfo\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.196748 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.196527    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt9f8\" (UniqueName: \"kubernetes.io/projected/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-kube-api-access-vt9f8\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.196748 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.196643    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-credential-socket\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.196748 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.196674    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istio-token\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.196748 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.196735    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istio-data\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.196965 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.196752    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istiod-ca-cert\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.196965 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.196786    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-workload-certs\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.196965 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.196866    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-workload-socket\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.197170 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.197031    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-credential-socket\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.197234 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.197197    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-workload-socket\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.197292 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.197241    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istio-data\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.197343 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.197307    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-workload-certs\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.197571 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.197549    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istiod-ca-cert\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.198900 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.198872    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istio-envoy\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.199025 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.199015    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istio-podinfo\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.205401 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.205375    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt9f8\" (UniqueName: \"kubernetes.io/projected/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-kube-api-access-vt9f8\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.205512 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.205459    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istio-token\") pod \"e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") " pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.321234 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.321199    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:49.454063 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.454034    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"]
Jul 20 21:44:49.455405 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:44:49.455373    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode97f02c7_7de4_4b38_a87b_7c9c903ae89f.slice/crio-b0c86ea49d67592fb889302feb57056c84c86b3438bbe744d08bfc5050b0d19a WatchSource:0}: Error finding container b0c86ea49d67592fb889302feb57056c84c86b3438bbe744d08bfc5050b0d19a: Status 404 returned error can't find the container with id b0c86ea49d67592fb889302feb57056c84c86b3438bbe744d08bfc5050b0d19a
Jul 20 21:44:49.457715 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.457686    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:44:49.457776 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.457750    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:44:49.457819 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:49.457786    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:44:50.048139 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:50.048102    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j" event={"ID":"e97f02c7-7de4-4b38-a87b-7c9c903ae89f","Type":"ContainerStarted","Data":"b130ac37c8fcb062da45f22295149bf085d69239152440df1b7ad311a7917718"}
Jul 20 21:44:50.048529 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:50.048159    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j" event={"ID":"e97f02c7-7de4-4b38-a87b-7c9c903ae89f","Type":"ContainerStarted","Data":"b0c86ea49d67592fb889302feb57056c84c86b3438bbe744d08bfc5050b0d19a"}
Jul 20 21:44:50.073666 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:50.073609    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j" podStartSLOduration=2.073589384 podStartE2EDuration="2.073589384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:44:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:44:50.071304595 +0000 UTC m=+2530.975294176" watchObservedRunningTime="2026-07-20 21:44:50.073589384 +0000 UTC m=+2530.977578963"
Jul 20 21:44:50.321936 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:50.321849    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:50.326690 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:50.326666    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:51.053006 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:51.052970    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:51.053866 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:51.053848    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:44:51.392023 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:44:51.391942    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:45:07.095329 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:07.095279    2561 patch_prober.go:28] interesting pod/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 20 21:45:07.095802 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:07.095362    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j" podUID="e97f02c7-7de4-4b38-a87b-7c9c903ae89f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 20 21:45:09.099994 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:09.099965    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:45:09.144363 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:09.144331    2561 generic.go:356] "Generic (PLEG): container finished" podID="e97f02c7-7de4-4b38-a87b-7c9c903ae89f" containerID="b130ac37c8fcb062da45f22295149bf085d69239152440df1b7ad311a7917718" exitCode=0
Jul 20 21:45:09.144519 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:09.144406    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j" event={"ID":"e97f02c7-7de4-4b38-a87b-7c9c903ae89f","Type":"ContainerDied","Data":"b130ac37c8fcb062da45f22295149bf085d69239152440df1b7ad311a7917718"}
Jul 20 21:45:09.144812 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:09.144797    2561 scope.go:117] "RemoveContainer" containerID="b130ac37c8fcb062da45f22295149bf085d69239152440df1b7ad311a7917718"
Jul 20 21:45:09.145755 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:09.145728    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:45:09.145859 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:09.145826    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:45:09.145908 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:09.145862    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 20 21:45:10.154604 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:10.154559    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j" event={"ID":"e97f02c7-7de4-4b38-a87b-7c9c903ae89f","Type":"ContainerStarted","Data":"60d718106514ce517914f7b56841caca90113aa773cb0cd6378f5da6f249d6a8"}
Jul 20 21:45:10.321894 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:10.321865    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:45:10.322138 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:10.322114    2561 patch_prober.go:28] interesting pod/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 20 21:45:10.322199 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:10.322176    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j" podUID="e97f02c7-7de4-4b38-a87b-7c9c903ae89f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 20 21:45:11.321794 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:11.321758    2561 patch_prober.go:28] interesting pod/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 20 21:45:11.322218 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:11.321826    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j" podUID="e97f02c7-7de4-4b38-a87b-7c9c903ae89f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 20 21:45:12.322162 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:12.322125    2561 patch_prober.go:28] interesting pod/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 20 21:45:12.322571 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:12.322195    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j" podUID="e97f02c7-7de4-4b38-a87b-7c9c903ae89f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 20 21:45:13.322443 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:13.322408    2561 patch_prober.go:28] interesting pod/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 20 21:45:13.322884 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:13.322467    2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j" podUID="e97f02c7-7de4-4b38-a87b-7c9c903ae89f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 20 21:45:14.325150 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:14.325119    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:45:14.325524 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:14.325388    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:45:14.326161 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:14.326144    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:45:24.992529 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:24.992494    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:45:51.584206 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:51.584167    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:45:58.585781 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:58.585734    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"]
Jul 20 21:45:58.586289 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:58.585992    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j" podUID="e97f02c7-7de4-4b38-a87b-7c9c903ae89f" containerName="istio-proxy" containerID="cri-o://60d718106514ce517914f7b56841caca90113aa773cb0cd6378f5da6f249d6a8" gracePeriod=30
Jul 20 21:45:59.327617 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:59.327576    2561 patch_prober.go:28] interesting pod/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 20 21:45:59.327785 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:45:59.327629    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j" podUID="e97f02c7-7de4-4b38-a87b-7c9c903ae89f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 20 21:46:03.830311 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.830288    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:46:03.869055 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.868979    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istio-podinfo\") pod \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") "
Jul 20 21:46:03.869055 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.869031    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vt9f8\" (UniqueName: \"kubernetes.io/projected/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-kube-api-access-vt9f8\") pod \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") "
Jul 20 21:46:03.869253 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.869104    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-workload-socket\") pod \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") "
Jul 20 21:46:03.869253 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.869127    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-workload-certs\") pod \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") "
Jul 20 21:46:03.869253 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.869187    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istio-envoy\") pod \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") "
Jul 20 21:46:03.869253 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.869212    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istio-data\") pod \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") "
Jul 20 21:46:03.869473 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.869256    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-credential-socket\") pod \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") "
Jul 20 21:46:03.869473 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.869283    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istio-token\") pod \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") "
Jul 20 21:46:03.869473 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.869314    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istiod-ca-cert\") pod \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\" (UID: \"e97f02c7-7de4-4b38-a87b-7c9c903ae89f\") "
Jul 20 21:46:03.869473 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.869386    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e97f02c7-7de4-4b38-a87b-7c9c903ae89f" (UID: "e97f02c7-7de4-4b38-a87b-7c9c903ae89f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:46:03.869473 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.869423    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e97f02c7-7de4-4b38-a87b-7c9c903ae89f" (UID: "e97f02c7-7de4-4b38-a87b-7c9c903ae89f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:46:03.869746 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.869542    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e97f02c7-7de4-4b38-a87b-7c9c903ae89f" (UID: "e97f02c7-7de4-4b38-a87b-7c9c903ae89f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:46:03.869746 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.869711    2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-workload-socket\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:46:03.869746 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.869733    2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-workload-certs\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:46:03.869914 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.869738    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e97f02c7-7de4-4b38-a87b-7c9c903ae89f" (UID: "e97f02c7-7de4-4b38-a87b-7c9c903ae89f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 21:46:03.869914 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.869746    2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-credential-socket\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:46:03.870826 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.870795    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istio-data" (OuterVolumeSpecName: "istio-data") pod "e97f02c7-7de4-4b38-a87b-7c9c903ae89f" (UID: "e97f02c7-7de4-4b38-a87b-7c9c903ae89f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:46:03.871316 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.871285    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-kube-api-access-vt9f8" (OuterVolumeSpecName: "kube-api-access-vt9f8") pod "e97f02c7-7de4-4b38-a87b-7c9c903ae89f" (UID: "e97f02c7-7de4-4b38-a87b-7c9c903ae89f"). InnerVolumeSpecName "kube-api-access-vt9f8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 21:46:03.871316 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.871301    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e97f02c7-7de4-4b38-a87b-7c9c903ae89f" (UID: "e97f02c7-7de4-4b38-a87b-7c9c903ae89f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:46:03.871439 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.871379    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e97f02c7-7de4-4b38-a87b-7c9c903ae89f" (UID: "e97f02c7-7de4-4b38-a87b-7c9c903ae89f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 21:46:03.871439 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.871428    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istio-token" (OuterVolumeSpecName: "istio-token") pod "e97f02c7-7de4-4b38-a87b-7c9c903ae89f" (UID: "e97f02c7-7de4-4b38-a87b-7c9c903ae89f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 21:46:03.970231 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.970196    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vt9f8\" (UniqueName: \"kubernetes.io/projected/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-kube-api-access-vt9f8\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:46:03.970231 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.970224    2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istio-envoy\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:46:03.970231 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.970234    2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istio-data\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:46:03.970464 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.970242    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istio-token\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:46:03.970464 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.970250    2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istiod-ca-cert\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:46:03.970464 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:03.970257    2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e97f02c7-7de4-4b38-a87b-7c9c903ae89f-istio-podinfo\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:46:04.413410 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:04.413368    2561 generic.go:356] "Generic (PLEG): container finished" podID="e97f02c7-7de4-4b38-a87b-7c9c903ae89f" containerID="60d718106514ce517914f7b56841caca90113aa773cb0cd6378f5da6f249d6a8" exitCode=0
Jul 20 21:46:04.413590 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:04.413437    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"
Jul 20 21:46:04.413590 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:04.413455    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j" event={"ID":"e97f02c7-7de4-4b38-a87b-7c9c903ae89f","Type":"ContainerDied","Data":"60d718106514ce517914f7b56841caca90113aa773cb0cd6378f5da6f249d6a8"}
Jul 20 21:46:04.413590 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:04.413548    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j" event={"ID":"e97f02c7-7de4-4b38-a87b-7c9c903ae89f","Type":"ContainerDied","Data":"b0c86ea49d67592fb889302feb57056c84c86b3438bbe744d08bfc5050b0d19a"}
Jul 20 21:46:04.413590 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:04.413572    2561 scope.go:117] "RemoveContainer" containerID="60d718106514ce517914f7b56841caca90113aa773cb0cd6378f5da6f249d6a8"
Jul 20 21:46:04.423266 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:04.423245    2561 scope.go:117] "RemoveContainer" containerID="b130ac37c8fcb062da45f22295149bf085d69239152440df1b7ad311a7917718"
Jul 20 21:46:04.431525 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:04.431508    2561 scope.go:117] "RemoveContainer" containerID="60d718106514ce517914f7b56841caca90113aa773cb0cd6378f5da6f249d6a8"
Jul 20 21:46:04.431738 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:46:04.431720    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60d718106514ce517914f7b56841caca90113aa773cb0cd6378f5da6f249d6a8\": container with ID starting with 60d718106514ce517914f7b56841caca90113aa773cb0cd6378f5da6f249d6a8 not found: ID does not exist" containerID="60d718106514ce517914f7b56841caca90113aa773cb0cd6378f5da6f249d6a8"
Jul 20 21:46:04.431784 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:04.431747    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60d718106514ce517914f7b56841caca90113aa773cb0cd6378f5da6f249d6a8"} err="failed to get container status \"60d718106514ce517914f7b56841caca90113aa773cb0cd6378f5da6f249d6a8\": rpc error: code = NotFound desc = could not find container \"60d718106514ce517914f7b56841caca90113aa773cb0cd6378f5da6f249d6a8\": container with ID starting with 60d718106514ce517914f7b56841caca90113aa773cb0cd6378f5da6f249d6a8 not found: ID does not exist"
Jul 20 21:46:04.431784 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:04.431765    2561 scope.go:117] "RemoveContainer" containerID="b130ac37c8fcb062da45f22295149bf085d69239152440df1b7ad311a7917718"
Jul 20 21:46:04.432026 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:46:04.432006    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b130ac37c8fcb062da45f22295149bf085d69239152440df1b7ad311a7917718\": container with ID starting with b130ac37c8fcb062da45f22295149bf085d69239152440df1b7ad311a7917718 not found: ID does not exist" containerID="b130ac37c8fcb062da45f22295149bf085d69239152440df1b7ad311a7917718"
Jul 20 21:46:04.432119 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:04.432034    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b130ac37c8fcb062da45f22295149bf085d69239152440df1b7ad311a7917718"} err="failed to get container status \"b130ac37c8fcb062da45f22295149bf085d69239152440df1b7ad311a7917718\": rpc error: code = NotFound desc = could not find container \"b130ac37c8fcb062da45f22295149bf085d69239152440df1b7ad311a7917718\": container with ID starting with b130ac37c8fcb062da45f22295149bf085d69239152440df1b7ad311a7917718 not found: ID does not exist"
Jul 20 21:46:04.453450 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:04.453422    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"]
Jul 20 21:46:04.461578 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:04.461557    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-08ebd1-openshift-default-7f87984945-44q6j"]
Jul 20 21:46:05.726162 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:05.726128    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e97f02c7-7de4-4b38-a87b-7c9c903ae89f" path="/var/lib/kubelet/pods/e97f02c7-7de4-4b38-a87b-7c9c903ae89f/volumes"
Jul 20 21:46:07.785582 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:07.785552    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:46:23.798655 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:23.798621    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:46:24.316645 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.316614    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"]
Jul 20 21:46:24.316947 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.316936    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e97f02c7-7de4-4b38-a87b-7c9c903ae89f" containerName="istio-proxy"
Jul 20 21:46:24.316999 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.316949    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e97f02c7-7de4-4b38-a87b-7c9c903ae89f" containerName="istio-proxy"
Jul 20 21:46:24.316999 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.316959    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e97f02c7-7de4-4b38-a87b-7c9c903ae89f" containerName="istio-proxy"
Jul 20 21:46:24.316999 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.316966    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e97f02c7-7de4-4b38-a87b-7c9c903ae89f" containerName="istio-proxy"
Jul 20 21:46:24.316999 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.316978    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95e61aaa-df4d-4d13-bb33-06d0e2aef4c3" containerName="istio-proxy"
Jul 20 21:46:24.316999 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.316985    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="95e61aaa-df4d-4d13-bb33-06d0e2aef4c3" containerName="istio-proxy"
Jul 20 21:46:24.317197 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.317036    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e97f02c7-7de4-4b38-a87b-7c9c903ae89f" containerName="istio-proxy"
Jul 20 21:46:24.317197 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.317044    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e97f02c7-7de4-4b38-a87b-7c9c903ae89f" containerName="istio-proxy"
Jul 20 21:46:24.320114 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.320092    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:46:24.326514 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.326336    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-196c99\"/\"kube-root-ca.crt\""
Jul 20 21:46:24.326514 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.326414    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-196c99\"/\"auth-test-model-196c99-kserve-self-signed-certs\""
Jul 20 21:46:24.326514 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.326336    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-196c99\"/\"default-dockercfg-28dv8\""
Jul 20 21:46:24.326673 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.326576    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-196c99\"/\"openshift-service-ca.crt\""
Jul 20 21:46:24.331739 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.331717    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"]
Jul 20 21:46:24.449925 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.449887    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a8a579e-ce5d-40bf-8e76-2832401015c9-tls-certs\") pod \"auth-test-model-196c99-kserve-84d87c4d79-kpj92\" (UID: \"3a8a579e-ce5d-40bf-8e76-2832401015c9\") " pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:46:24.449925 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.449927    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpc8f\" (UniqueName: \"kubernetes.io/projected/3a8a579e-ce5d-40bf-8e76-2832401015c9-kube-api-access-hpc8f\") pod \"auth-test-model-196c99-kserve-84d87c4d79-kpj92\" (UID: \"3a8a579e-ce5d-40bf-8e76-2832401015c9\") " pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:46:24.450186 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.449947    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a8a579e-ce5d-40bf-8e76-2832401015c9-home\") pod \"auth-test-model-196c99-kserve-84d87c4d79-kpj92\" (UID: \"3a8a579e-ce5d-40bf-8e76-2832401015c9\") " pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:46:24.450186 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.449970    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a8a579e-ce5d-40bf-8e76-2832401015c9-dshm\") pod \"auth-test-model-196c99-kserve-84d87c4d79-kpj92\" (UID: \"3a8a579e-ce5d-40bf-8e76-2832401015c9\") " pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:46:24.450186 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.450038    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a8a579e-ce5d-40bf-8e76-2832401015c9-tmp-dir\") pod \"auth-test-model-196c99-kserve-84d87c4d79-kpj92\" (UID: \"3a8a579e-ce5d-40bf-8e76-2832401015c9\") " pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:46:24.450186 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.450136    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a8a579e-ce5d-40bf-8e76-2832401015c9-model-cache\") pod \"auth-test-model-196c99-kserve-84d87c4d79-kpj92\" (UID: \"3a8a579e-ce5d-40bf-8e76-2832401015c9\") " pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:46:24.551257 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.551221    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a8a579e-ce5d-40bf-8e76-2832401015c9-tls-certs\") pod \"auth-test-model-196c99-kserve-84d87c4d79-kpj92\" (UID: \"3a8a579e-ce5d-40bf-8e76-2832401015c9\") " pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:46:24.551257 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.551262    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpc8f\" (UniqueName: \"kubernetes.io/projected/3a8a579e-ce5d-40bf-8e76-2832401015c9-kube-api-access-hpc8f\") pod \"auth-test-model-196c99-kserve-84d87c4d79-kpj92\" (UID: \"3a8a579e-ce5d-40bf-8e76-2832401015c9\") " pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:46:24.551530 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.551283    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a8a579e-ce5d-40bf-8e76-2832401015c9-home\") pod \"auth-test-model-196c99-kserve-84d87c4d79-kpj92\" (UID: \"3a8a579e-ce5d-40bf-8e76-2832401015c9\") " pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:46:24.551530 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.551310    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a8a579e-ce5d-40bf-8e76-2832401015c9-dshm\") pod \"auth-test-model-196c99-kserve-84d87c4d79-kpj92\" (UID: \"3a8a579e-ce5d-40bf-8e76-2832401015c9\") " pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:46:24.551530 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.551353    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a8a579e-ce5d-40bf-8e76-2832401015c9-tmp-dir\") pod \"auth-test-model-196c99-kserve-84d87c4d79-kpj92\" (UID: \"3a8a579e-ce5d-40bf-8e76-2832401015c9\") " pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:46:24.551530 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.551411    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a8a579e-ce5d-40bf-8e76-2832401015c9-model-cache\") pod \"auth-test-model-196c99-kserve-84d87c4d79-kpj92\" (UID: \"3a8a579e-ce5d-40bf-8e76-2832401015c9\") " pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:46:24.551754 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.551720    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a8a579e-ce5d-40bf-8e76-2832401015c9-home\") pod \"auth-test-model-196c99-kserve-84d87c4d79-kpj92\" (UID: \"3a8a579e-ce5d-40bf-8e76-2832401015c9\") " pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:46:24.551803 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.551786    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a8a579e-ce5d-40bf-8e76-2832401015c9-tmp-dir\") pod \"auth-test-model-196c99-kserve-84d87c4d79-kpj92\" (UID: \"3a8a579e-ce5d-40bf-8e76-2832401015c9\") " pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:46:24.551842 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.551818    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a8a579e-ce5d-40bf-8e76-2832401015c9-model-cache\") pod \"auth-test-model-196c99-kserve-84d87c4d79-kpj92\" (UID: \"3a8a579e-ce5d-40bf-8e76-2832401015c9\") " pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:46:24.553603 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.553580    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a8a579e-ce5d-40bf-8e76-2832401015c9-dshm\") pod \"auth-test-model-196c99-kserve-84d87c4d79-kpj92\" (UID: \"3a8a579e-ce5d-40bf-8e76-2832401015c9\") " pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:46:24.553866 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.553848    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a8a579e-ce5d-40bf-8e76-2832401015c9-tls-certs\") pod \"auth-test-model-196c99-kserve-84d87c4d79-kpj92\" (UID: \"3a8a579e-ce5d-40bf-8e76-2832401015c9\") " pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:46:24.560515 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.560490    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpc8f\" (UniqueName: \"kubernetes.io/projected/3a8a579e-ce5d-40bf-8e76-2832401015c9-kube-api-access-hpc8f\") pod \"auth-test-model-196c99-kserve-84d87c4d79-kpj92\" (UID: \"3a8a579e-ce5d-40bf-8e76-2832401015c9\") " pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:46:24.629105 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.628983    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:46:24.768168 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:24.768145    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"]
Jul 20 21:46:24.770657 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:46:24.770626    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a8a579e_ce5d_40bf_8e76_2832401015c9.slice/crio-5f9a4d7a7e4de5d60b0b2aba09e59d85e41a9997b34a271db4ca6457ae3edc22 WatchSource:0}: Error finding container 5f9a4d7a7e4de5d60b0b2aba09e59d85e41a9997b34a271db4ca6457ae3edc22: Status 404 returned error can't find the container with id 5f9a4d7a7e4de5d60b0b2aba09e59d85e41a9997b34a271db4ca6457ae3edc22
Jul 20 21:46:25.516905 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:25.516869    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92" event={"ID":"3a8a579e-ce5d-40bf-8e76-2832401015c9","Type":"ContainerStarted","Data":"833f143ad9aba83c234a1818d818286e95c63756d91dd369531cb5b1566c3daf"}
Jul 20 21:46:25.516905 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:25.516910    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92" event={"ID":"3a8a579e-ce5d-40bf-8e76-2832401015c9","Type":"ContainerStarted","Data":"5f9a4d7a7e4de5d60b0b2aba09e59d85e41a9997b34a271db4ca6457ae3edc22"}
Jul 20 21:46:25.535623 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:25.535570    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92" podStartSLOduration=1.5355542359999999 podStartE2EDuration="1.535554236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:46:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:46:25.533955096 +0000 UTC m=+2626.437944675" watchObservedRunningTime="2026-07-20 21:46:25.535554236 +0000 UTC m=+2626.439543814"
Jul 20 21:46:34.629889 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:34.629855    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:46:34.629889 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:34.629907    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:46:34.642436 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:34.642410    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:46:35.002532 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:35.002442    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:46:35.576012 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:35.575978    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:46:39.916786 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:39.916749    2561 scope.go:117] "RemoveContainer" containerID="ba9676958210a11e4ed76dbb42d8151e444eeade9acbcf7a560168c302eab967"
Jul 20 21:46:46.690335 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:46:46.690300    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:47:04.792680 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:47:04.792641    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:47:20.205996 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:47:20.205957    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:47:38.382723 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:47:38.382637    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:47:40.541502 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:47:40.541474    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/1.log"
Jul 20 21:47:40.549117 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:47:40.549093    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/1.log"
Jul 20 21:47:54.913441 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:47:54.913401    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:48:12.996768 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:48:12.996729    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:48:31.095857 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:48:31.095818    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:48:48.299034 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:48:48.298998    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:48:51.967014 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:48:51.966975    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"]
Jul 20 21:49:05.889925 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:05.889846    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:49:06.986667 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:06.986612    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92" podUID="3a8a579e-ce5d-40bf-8e76-2832401015c9" containerName="main" containerID="cri-o://833f143ad9aba83c234a1818d818286e95c63756d91dd369531cb5b1566c3daf" gracePeriod=45
Jul 20 21:49:07.232915 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.232892    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:49:07.280576 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.280500    2561 generic.go:356] "Generic (PLEG): container finished" podID="3a8a579e-ce5d-40bf-8e76-2832401015c9" containerID="833f143ad9aba83c234a1818d818286e95c63756d91dd369531cb5b1566c3daf" exitCode=0
Jul 20 21:49:07.280576 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.280539    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"
Jul 20 21:49:07.280784 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.280539    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92" event={"ID":"3a8a579e-ce5d-40bf-8e76-2832401015c9","Type":"ContainerDied","Data":"833f143ad9aba83c234a1818d818286e95c63756d91dd369531cb5b1566c3daf"}
Jul 20 21:49:07.280784 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.280655    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92" event={"ID":"3a8a579e-ce5d-40bf-8e76-2832401015c9","Type":"ContainerDied","Data":"5f9a4d7a7e4de5d60b0b2aba09e59d85e41a9997b34a271db4ca6457ae3edc22"}
Jul 20 21:49:07.280784 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.280678    2561 scope.go:117] "RemoveContainer" containerID="833f143ad9aba83c234a1818d818286e95c63756d91dd369531cb5b1566c3daf"
Jul 20 21:49:07.288975 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.288957    2561 scope.go:117] "RemoveContainer" containerID="833f143ad9aba83c234a1818d818286e95c63756d91dd369531cb5b1566c3daf"
Jul 20 21:49:07.289247 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:49:07.289225    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"833f143ad9aba83c234a1818d818286e95c63756d91dd369531cb5b1566c3daf\": container with ID starting with 833f143ad9aba83c234a1818d818286e95c63756d91dd369531cb5b1566c3daf not found: ID does not exist" containerID="833f143ad9aba83c234a1818d818286e95c63756d91dd369531cb5b1566c3daf"
Jul 20 21:49:07.289306 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.289256    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"833f143ad9aba83c234a1818d818286e95c63756d91dd369531cb5b1566c3daf"} err="failed to get container status \"833f143ad9aba83c234a1818d818286e95c63756d91dd369531cb5b1566c3daf\": rpc error: code = NotFound desc = could not find container \"833f143ad9aba83c234a1818d818286e95c63756d91dd369531cb5b1566c3daf\": container with ID starting with 833f143ad9aba83c234a1818d818286e95c63756d91dd369531cb5b1566c3daf not found: ID does not exist"
Jul 20 21:49:07.299913 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.299896    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a8a579e-ce5d-40bf-8e76-2832401015c9-model-cache\") pod \"3a8a579e-ce5d-40bf-8e76-2832401015c9\" (UID: \"3a8a579e-ce5d-40bf-8e76-2832401015c9\") "
Jul 20 21:49:07.299982 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.299926    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a8a579e-ce5d-40bf-8e76-2832401015c9-dshm\") pod \"3a8a579e-ce5d-40bf-8e76-2832401015c9\" (UID: \"3a8a579e-ce5d-40bf-8e76-2832401015c9\") "
Jul 20 21:49:07.299982 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.299961    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a8a579e-ce5d-40bf-8e76-2832401015c9-home\") pod \"3a8a579e-ce5d-40bf-8e76-2832401015c9\" (UID: \"3a8a579e-ce5d-40bf-8e76-2832401015c9\") "
Jul 20 21:49:07.300085 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.299993    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a8a579e-ce5d-40bf-8e76-2832401015c9-tls-certs\") pod \"3a8a579e-ce5d-40bf-8e76-2832401015c9\" (UID: \"3a8a579e-ce5d-40bf-8e76-2832401015c9\") "
Jul 20 21:49:07.300085 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.300010    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpc8f\" (UniqueName: \"kubernetes.io/projected/3a8a579e-ce5d-40bf-8e76-2832401015c9-kube-api-access-hpc8f\") pod \"3a8a579e-ce5d-40bf-8e76-2832401015c9\" (UID: \"3a8a579e-ce5d-40bf-8e76-2832401015c9\") "
Jul 20 21:49:07.300085 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.300033    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a8a579e-ce5d-40bf-8e76-2832401015c9-tmp-dir\") pod \"3a8a579e-ce5d-40bf-8e76-2832401015c9\" (UID: \"3a8a579e-ce5d-40bf-8e76-2832401015c9\") "
Jul 20 21:49:07.300238 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.300132    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a8a579e-ce5d-40bf-8e76-2832401015c9-model-cache" (OuterVolumeSpecName: "model-cache") pod "3a8a579e-ce5d-40bf-8e76-2832401015c9" (UID: "3a8a579e-ce5d-40bf-8e76-2832401015c9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:49:07.300307 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.300275    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a8a579e-ce5d-40bf-8e76-2832401015c9-home" (OuterVolumeSpecName: "home") pod "3a8a579e-ce5d-40bf-8e76-2832401015c9" (UID: "3a8a579e-ce5d-40bf-8e76-2832401015c9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:49:07.300364 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.300346    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a8a579e-ce5d-40bf-8e76-2832401015c9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3a8a579e-ce5d-40bf-8e76-2832401015c9" (UID: "3a8a579e-ce5d-40bf-8e76-2832401015c9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:49:07.300476 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.300459    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a8a579e-ce5d-40bf-8e76-2832401015c9-home\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:49:07.300532 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.300480    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a8a579e-ce5d-40bf-8e76-2832401015c9-tmp-dir\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:49:07.300532 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.300493    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a8a579e-ce5d-40bf-8e76-2832401015c9-model-cache\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:49:07.302059 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.302037    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a8a579e-ce5d-40bf-8e76-2832401015c9-dshm" (OuterVolumeSpecName: "dshm") pod "3a8a579e-ce5d-40bf-8e76-2832401015c9" (UID: "3a8a579e-ce5d-40bf-8e76-2832401015c9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:49:07.302059 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.302042    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a8a579e-ce5d-40bf-8e76-2832401015c9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3a8a579e-ce5d-40bf-8e76-2832401015c9" (UID: "3a8a579e-ce5d-40bf-8e76-2832401015c9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 21:49:07.302197 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.302101    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a8a579e-ce5d-40bf-8e76-2832401015c9-kube-api-access-hpc8f" (OuterVolumeSpecName: "kube-api-access-hpc8f") pod "3a8a579e-ce5d-40bf-8e76-2832401015c9" (UID: "3a8a579e-ce5d-40bf-8e76-2832401015c9"). InnerVolumeSpecName "kube-api-access-hpc8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 21:49:07.401757 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.401730    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a8a579e-ce5d-40bf-8e76-2832401015c9-dshm\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:49:07.401757 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.401753    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a8a579e-ce5d-40bf-8e76-2832401015c9-tls-certs\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:49:07.401948 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.401765    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpc8f\" (UniqueName: \"kubernetes.io/projected/3a8a579e-ce5d-40bf-8e76-2832401015c9-kube-api-access-hpc8f\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:49:07.607430 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.607403    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"]
Jul 20 21:49:07.611918 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.611888    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-196c99/auth-test-model-196c99-kserve-84d87c4d79-kpj92"]
Jul 20 21:49:07.726495 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:07.726458    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a8a579e-ce5d-40bf-8e76-2832401015c9" path="/var/lib/kubelet/pods/3a8a579e-ce5d-40bf-8e76-2832401015c9/volumes"
Jul 20 21:49:19.302447 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.302411    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"]
Jul 20 21:49:19.303057 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.302898    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a8a579e-ce5d-40bf-8e76-2832401015c9" containerName="main"
Jul 20 21:49:19.303057 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.302913    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a8a579e-ce5d-40bf-8e76-2832401015c9" containerName="main"
Jul 20 21:49:19.303057 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.302999    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a8a579e-ce5d-40bf-8e76-2832401015c9" containerName="main"
Jul 20 21:49:19.305984 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.305952    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:49:19.308908 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.308888    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-196c99\"/\"kube-root-ca.crt\""
Jul 20 21:49:19.309669 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.309646    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-196c99\"/\"default-dockercfg-28dv8\""
Jul 20 21:49:19.309776 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.309655    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-196c99\"/\"openshift-service-ca.crt\""
Jul 20 21:49:19.309776 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.309697    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-196c99\"/\"sub-test-model-196c99-kserve-self-signed-certs\""
Jul 20 21:49:19.314458 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.314415    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"]
Jul 20 21:49:19.405122 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.405068    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzz6k\" (UniqueName: \"kubernetes.io/projected/798196e4-ac57-40e4-b1f9-cf5d83cf7195-kube-api-access-tzz6k\") pod \"sub-test-model-196c99-kserve-968469945-rc5f2\" (UID: \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\") " pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:49:19.405294 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.405140    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/798196e4-ac57-40e4-b1f9-cf5d83cf7195-dshm\") pod \"sub-test-model-196c99-kserve-968469945-rc5f2\" (UID: \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\") " pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:49:19.405294 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.405176    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/798196e4-ac57-40e4-b1f9-cf5d83cf7195-home\") pod \"sub-test-model-196c99-kserve-968469945-rc5f2\" (UID: \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\") " pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:49:19.405294 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.405196    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/798196e4-ac57-40e4-b1f9-cf5d83cf7195-tmp-dir\") pod \"sub-test-model-196c99-kserve-968469945-rc5f2\" (UID: \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\") " pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:49:19.405417 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.405308    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/798196e4-ac57-40e4-b1f9-cf5d83cf7195-model-cache\") pod \"sub-test-model-196c99-kserve-968469945-rc5f2\" (UID: \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\") " pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:49:19.405417 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.405377    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/798196e4-ac57-40e4-b1f9-cf5d83cf7195-tls-certs\") pod \"sub-test-model-196c99-kserve-968469945-rc5f2\" (UID: \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\") " pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:49:19.506589 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.506559    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/798196e4-ac57-40e4-b1f9-cf5d83cf7195-dshm\") pod \"sub-test-model-196c99-kserve-968469945-rc5f2\" (UID: \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\") " pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:49:19.506770 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.506597    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/798196e4-ac57-40e4-b1f9-cf5d83cf7195-home\") pod \"sub-test-model-196c99-kserve-968469945-rc5f2\" (UID: \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\") " pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:49:19.506770 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.506613    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/798196e4-ac57-40e4-b1f9-cf5d83cf7195-tmp-dir\") pod \"sub-test-model-196c99-kserve-968469945-rc5f2\" (UID: \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\") " pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:49:19.506770 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.506655    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/798196e4-ac57-40e4-b1f9-cf5d83cf7195-model-cache\") pod \"sub-test-model-196c99-kserve-968469945-rc5f2\" (UID: \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\") " pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:49:19.506770 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.506686    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/798196e4-ac57-40e4-b1f9-cf5d83cf7195-tls-certs\") pod \"sub-test-model-196c99-kserve-968469945-rc5f2\" (UID: \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\") " pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:49:19.506770 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.506705    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzz6k\" (UniqueName: \"kubernetes.io/projected/798196e4-ac57-40e4-b1f9-cf5d83cf7195-kube-api-access-tzz6k\") pod \"sub-test-model-196c99-kserve-968469945-rc5f2\" (UID: \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\") " pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:49:19.507032 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.507006    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/798196e4-ac57-40e4-b1f9-cf5d83cf7195-home\") pod \"sub-test-model-196c99-kserve-968469945-rc5f2\" (UID: \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\") " pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:49:19.507106 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.507044    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/798196e4-ac57-40e4-b1f9-cf5d83cf7195-model-cache\") pod \"sub-test-model-196c99-kserve-968469945-rc5f2\" (UID: \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\") " pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:49:19.507177 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.507160    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/798196e4-ac57-40e4-b1f9-cf5d83cf7195-tmp-dir\") pod \"sub-test-model-196c99-kserve-968469945-rc5f2\" (UID: \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\") " pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:49:19.508884 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.508862    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/798196e4-ac57-40e4-b1f9-cf5d83cf7195-dshm\") pod \"sub-test-model-196c99-kserve-968469945-rc5f2\" (UID: \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\") " pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:49:19.509052 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.509038    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/798196e4-ac57-40e4-b1f9-cf5d83cf7195-tls-certs\") pod \"sub-test-model-196c99-kserve-968469945-rc5f2\" (UID: \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\") " pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:49:19.514839 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.514817    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzz6k\" (UniqueName: \"kubernetes.io/projected/798196e4-ac57-40e4-b1f9-cf5d83cf7195-kube-api-access-tzz6k\") pod \"sub-test-model-196c99-kserve-968469945-rc5f2\" (UID: \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\") " pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:49:19.616654 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.616571    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:49:19.751585 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:19.751500    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"]
Jul 20 21:49:19.753725 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:49:19.753696    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod798196e4_ac57_40e4_b1f9_cf5d83cf7195.slice/crio-6ff792fad9fa6c53d124d9541ab9601de4e2f7f119567fc7eeccadbd73fef71a WatchSource:0}: Error finding container 6ff792fad9fa6c53d124d9541ab9601de4e2f7f119567fc7eeccadbd73fef71a: Status 404 returned error can't find the container with id 6ff792fad9fa6c53d124d9541ab9601de4e2f7f119567fc7eeccadbd73fef71a
Jul 20 21:49:20.341263 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:20.341220    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2" event={"ID":"798196e4-ac57-40e4-b1f9-cf5d83cf7195","Type":"ContainerStarted","Data":"852e728525a83815b448f0c012417cbad5bec7773fc6564f3a36e042fa100625"}
Jul 20 21:49:20.341696 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:20.341271    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2" event={"ID":"798196e4-ac57-40e4-b1f9-cf5d83cf7195","Type":"ContainerStarted","Data":"6ff792fad9fa6c53d124d9541ab9601de4e2f7f119567fc7eeccadbd73fef71a"}
Jul 20 21:49:20.358718 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:20.358669    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2" podStartSLOduration=1.358655922 podStartE2EDuration="1.358655922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:49:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:49:20.357338063 +0000 UTC m=+2801.261327642" watchObservedRunningTime="2026-07-20 21:49:20.358655922 +0000 UTC m=+2801.262645499"
Jul 20 21:49:23.287324 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:23.287289    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:49:29.617714 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:29.617671    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:49:29.618159 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:29.617728    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:49:29.630027 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:29.630003    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:49:30.399742 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:30.399711    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:49:49.599646 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:49:49.599599    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:50:07.736225 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:07.736187    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:50:28.296047 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:28.296012    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:50:34.475260 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:34.475168    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"]
Jul 20 21:50:39.223121 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.223086    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"]
Jul 20 21:50:39.226584 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.226564    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:50:39.230505 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.230477    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-196c99\"/\"rate-test-model-196c99-kserve-self-signed-certs\""
Jul 20 21:50:39.249015 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.248985    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"]
Jul 20 21:50:39.310278 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.310232    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-model-cache\") pod \"rate-test-model-196c99-kserve-7dbc86ff9f-jpblk\" (UID: \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\") " pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:50:39.310278 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.310281    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-dshm\") pod \"rate-test-model-196c99-kserve-7dbc86ff9f-jpblk\" (UID: \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\") " pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:50:39.310527 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.310301    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-tls-certs\") pod \"rate-test-model-196c99-kserve-7dbc86ff9f-jpblk\" (UID: \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\") " pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:50:39.310527 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.310388    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-home\") pod \"rate-test-model-196c99-kserve-7dbc86ff9f-jpblk\" (UID: \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\") " pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:50:39.310527 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.310433    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dthn\" (UniqueName: \"kubernetes.io/projected/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-kube-api-access-2dthn\") pod \"rate-test-model-196c99-kserve-7dbc86ff9f-jpblk\" (UID: \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\") " pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:50:39.310527 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.310482    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-tmp-dir\") pod \"rate-test-model-196c99-kserve-7dbc86ff9f-jpblk\" (UID: \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\") " pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:50:39.411952 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.411900    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-home\") pod \"rate-test-model-196c99-kserve-7dbc86ff9f-jpblk\" (UID: \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\") " pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:50:39.412153 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.411984    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dthn\" (UniqueName: \"kubernetes.io/projected/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-kube-api-access-2dthn\") pod \"rate-test-model-196c99-kserve-7dbc86ff9f-jpblk\" (UID: \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\") " pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:50:39.412153 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.412040    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-tmp-dir\") pod \"rate-test-model-196c99-kserve-7dbc86ff9f-jpblk\" (UID: \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\") " pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:50:39.412153 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.412113    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-model-cache\") pod \"rate-test-model-196c99-kserve-7dbc86ff9f-jpblk\" (UID: \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\") " pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:50:39.412153 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.412138    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-dshm\") pod \"rate-test-model-196c99-kserve-7dbc86ff9f-jpblk\" (UID: \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\") " pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:50:39.412356 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.412167    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-tls-certs\") pod \"rate-test-model-196c99-kserve-7dbc86ff9f-jpblk\" (UID: \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\") " pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:50:39.412447 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.412426    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-home\") pod \"rate-test-model-196c99-kserve-7dbc86ff9f-jpblk\" (UID: \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\") " pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:50:39.412519 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.412465    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-tmp-dir\") pod \"rate-test-model-196c99-kserve-7dbc86ff9f-jpblk\" (UID: \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\") " pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:50:39.412519 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.412492    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-model-cache\") pod \"rate-test-model-196c99-kserve-7dbc86ff9f-jpblk\" (UID: \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\") " pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:50:39.414430 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.414407    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-dshm\") pod \"rate-test-model-196c99-kserve-7dbc86ff9f-jpblk\" (UID: \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\") " pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:50:39.414686 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.414667    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-tls-certs\") pod \"rate-test-model-196c99-kserve-7dbc86ff9f-jpblk\" (UID: \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\") " pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:50:39.424674 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.424652    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dthn\" (UniqueName: \"kubernetes.io/projected/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-kube-api-access-2dthn\") pod \"rate-test-model-196c99-kserve-7dbc86ff9f-jpblk\" (UID: \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\") " pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:50:39.535359 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.535326    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:50:39.663951 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.663912    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"]
Jul 20 21:50:39.665672 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:50:39.665645    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd4fd8e36_71c8_4c57_a3b7_5ed91ed11a90.slice/crio-86746db3b945088dad7c0b860aacdf3a2c685e899fd16a5de19522df5ef98914 WatchSource:0}: Error finding container 86746db3b945088dad7c0b860aacdf3a2c685e899fd16a5de19522df5ef98914: Status 404 returned error can't find the container with id 86746db3b945088dad7c0b860aacdf3a2c685e899fd16a5de19522df5ef98914
Jul 20 21:50:39.714157 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.714123    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk" event={"ID":"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90","Type":"ContainerStarted","Data":"86746db3b945088dad7c0b860aacdf3a2c685e899fd16a5de19522df5ef98914"}
Jul 20 21:50:39.964615 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:39.964532    2561 scope.go:117] "RemoveContainer" containerID="9c9878c72710c5df0cbf5b8610a8eb9c7007ae2c276ca66ac85e768794e4166d"
Jul 20 21:50:40.720029 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:40.719994    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk" event={"ID":"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90","Type":"ContainerStarted","Data":"7a3afc5f93c58e826cfe303f1133c1c448d1c9ebab50884672e73ff01ba6e34b"}
Jul 20 21:50:40.740526 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:40.740466    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk" podStartSLOduration=1.740450839 podStartE2EDuration="1.740450839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:50:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:50:40.738098099 +0000 UTC m=+2881.642087668" watchObservedRunningTime="2026-07-20 21:50:40.740450839 +0000 UTC m=+2881.644440417"
Jul 20 21:50:47.000182 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:47.000145    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:50:49.494731 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.494666    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2" podUID="798196e4-ac57-40e4-b1f9-cf5d83cf7195" containerName="main" containerID="cri-o://852e728525a83815b448f0c012417cbad5bec7773fc6564f3a36e042fa100625" gracePeriod=45
Jul 20 21:50:49.536335 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.536297    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:50:49.536488 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.536349    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:50:49.549014 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.548987    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:50:49.744590 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.744563    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:50:49.766901 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.766811    2561 generic.go:356] "Generic (PLEG): container finished" podID="798196e4-ac57-40e4-b1f9-cf5d83cf7195" containerID="852e728525a83815b448f0c012417cbad5bec7773fc6564f3a36e042fa100625" exitCode=0
Jul 20 21:50:49.766901 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.766865    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"
Jul 20 21:50:49.767164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.766899    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2" event={"ID":"798196e4-ac57-40e4-b1f9-cf5d83cf7195","Type":"ContainerDied","Data":"852e728525a83815b448f0c012417cbad5bec7773fc6564f3a36e042fa100625"}
Jul 20 21:50:49.767164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.766945    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2" event={"ID":"798196e4-ac57-40e4-b1f9-cf5d83cf7195","Type":"ContainerDied","Data":"6ff792fad9fa6c53d124d9541ab9601de4e2f7f119567fc7eeccadbd73fef71a"}
Jul 20 21:50:49.767164 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.766974    2561 scope.go:117] "RemoveContainer" containerID="852e728525a83815b448f0c012417cbad5bec7773fc6564f3a36e042fa100625"
Jul 20 21:50:49.776401 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.776382    2561 scope.go:117] "RemoveContainer" containerID="852e728525a83815b448f0c012417cbad5bec7773fc6564f3a36e042fa100625"
Jul 20 21:50:49.776724 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:50:49.776688    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"852e728525a83815b448f0c012417cbad5bec7773fc6564f3a36e042fa100625\": container with ID starting with 852e728525a83815b448f0c012417cbad5bec7773fc6564f3a36e042fa100625 not found: ID does not exist" containerID="852e728525a83815b448f0c012417cbad5bec7773fc6564f3a36e042fa100625"
Jul 20 21:50:49.776798 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.776739    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"852e728525a83815b448f0c012417cbad5bec7773fc6564f3a36e042fa100625"} err="failed to get container status \"852e728525a83815b448f0c012417cbad5bec7773fc6564f3a36e042fa100625\": rpc error: code = NotFound desc = could not find container \"852e728525a83815b448f0c012417cbad5bec7773fc6564f3a36e042fa100625\": container with ID starting with 852e728525a83815b448f0c012417cbad5bec7773fc6564f3a36e042fa100625 not found: ID does not exist"
Jul 20 21:50:49.780063 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.780041    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:50:49.804408 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.804377    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzz6k\" (UniqueName: \"kubernetes.io/projected/798196e4-ac57-40e4-b1f9-cf5d83cf7195-kube-api-access-tzz6k\") pod \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\" (UID: \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\") "
Jul 20 21:50:49.804608 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.804417    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/798196e4-ac57-40e4-b1f9-cf5d83cf7195-model-cache\") pod \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\" (UID: \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\") "
Jul 20 21:50:49.804608 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.804464    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/798196e4-ac57-40e4-b1f9-cf5d83cf7195-dshm\") pod \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\" (UID: \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\") "
Jul 20 21:50:49.804608 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.804499    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/798196e4-ac57-40e4-b1f9-cf5d83cf7195-tls-certs\") pod \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\" (UID: \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\") "
Jul 20 21:50:49.804608 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.804560    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/798196e4-ac57-40e4-b1f9-cf5d83cf7195-home\") pod \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\" (UID: \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\") "
Jul 20 21:50:49.804608 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.804603    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/798196e4-ac57-40e4-b1f9-cf5d83cf7195-tmp-dir\") pod \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\" (UID: \"798196e4-ac57-40e4-b1f9-cf5d83cf7195\") "
Jul 20 21:50:49.804855 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.804743    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/798196e4-ac57-40e4-b1f9-cf5d83cf7195-model-cache" (OuterVolumeSpecName: "model-cache") pod "798196e4-ac57-40e4-b1f9-cf5d83cf7195" (UID: "798196e4-ac57-40e4-b1f9-cf5d83cf7195"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:50:49.804855 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.804825    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/798196e4-ac57-40e4-b1f9-cf5d83cf7195-home" (OuterVolumeSpecName: "home") pod "798196e4-ac57-40e4-b1f9-cf5d83cf7195" (UID: "798196e4-ac57-40e4-b1f9-cf5d83cf7195"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:50:49.804968 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.804851    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/798196e4-ac57-40e4-b1f9-cf5d83cf7195-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "798196e4-ac57-40e4-b1f9-cf5d83cf7195" (UID: "798196e4-ac57-40e4-b1f9-cf5d83cf7195"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:50:49.804968 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.804925    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/798196e4-ac57-40e4-b1f9-cf5d83cf7195-model-cache\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:50:49.804968 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.804940    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/798196e4-ac57-40e4-b1f9-cf5d83cf7195-home\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:50:49.804968 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.804953    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/798196e4-ac57-40e4-b1f9-cf5d83cf7195-tmp-dir\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:50:49.806579 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.806558    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/798196e4-ac57-40e4-b1f9-cf5d83cf7195-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "798196e4-ac57-40e4-b1f9-cf5d83cf7195" (UID: "798196e4-ac57-40e4-b1f9-cf5d83cf7195"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 21:50:49.807023 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.806999    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/798196e4-ac57-40e4-b1f9-cf5d83cf7195-dshm" (OuterVolumeSpecName: "dshm") pod "798196e4-ac57-40e4-b1f9-cf5d83cf7195" (UID: "798196e4-ac57-40e4-b1f9-cf5d83cf7195"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:50:49.807189 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.807062    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/798196e4-ac57-40e4-b1f9-cf5d83cf7195-kube-api-access-tzz6k" (OuterVolumeSpecName: "kube-api-access-tzz6k") pod "798196e4-ac57-40e4-b1f9-cf5d83cf7195" (UID: "798196e4-ac57-40e4-b1f9-cf5d83cf7195"). InnerVolumeSpecName "kube-api-access-tzz6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 21:50:49.906391 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.906344    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzz6k\" (UniqueName: \"kubernetes.io/projected/798196e4-ac57-40e4-b1f9-cf5d83cf7195-kube-api-access-tzz6k\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:50:49.906391 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.906382    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/798196e4-ac57-40e4-b1f9-cf5d83cf7195-dshm\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:50:49.906391 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:49.906391    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/798196e4-ac57-40e4-b1f9-cf5d83cf7195-tls-certs\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:50:50.091376 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:50.091346    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"]
Jul 20 21:50:50.095601 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:50.095575    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-196c99/sub-test-model-196c99-kserve-968469945-rc5f2"]
Jul 20 21:50:51.725602 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:51.725567    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="798196e4-ac57-40e4-b1f9-cf5d83cf7195" path="/var/lib/kubelet/pods/798196e4-ac57-40e4-b1f9-cf5d83cf7195/volumes"
Jul 20 21:50:56.698414 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.698368    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"]
Jul 20 21:50:56.698912 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.698895    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="798196e4-ac57-40e4-b1f9-cf5d83cf7195" containerName="main"
Jul 20 21:50:56.698982 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.698913    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="798196e4-ac57-40e4-b1f9-cf5d83cf7195" containerName="main"
Jul 20 21:50:56.698982 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.698971    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="798196e4-ac57-40e4-b1f9-cf5d83cf7195" containerName="main"
Jul 20 21:50:56.702976 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.702959    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:50:56.706554 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.706536    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-046c73\"/\"kube-root-ca.crt\""
Jul 20 21:50:56.706662 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.706559    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-046c73\"/\"default-dockercfg-rwqs9\""
Jul 20 21:50:56.706662 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.706578    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-046c73\"/\"openshift-service-ca.crt\""
Jul 20 21:50:56.706662 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.706560    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-046c73\"/\"rate-test-model-046c73-kserve-self-signed-certs\""
Jul 20 21:50:56.710934 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.710913    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"]
Jul 20 21:50:56.769082 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.769050    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6865q\" (UniqueName: \"kubernetes.io/projected/b2531468-09bc-4f41-bbe8-4522f994fb8e-kube-api-access-6865q\") pod \"rate-test-model-046c73-kserve-54965f5575-cs8d5\" (UID: \"b2531468-09bc-4f41-bbe8-4522f994fb8e\") " pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:50:56.769248 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.769118    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2531468-09bc-4f41-bbe8-4522f994fb8e-home\") pod \"rate-test-model-046c73-kserve-54965f5575-cs8d5\" (UID: \"b2531468-09bc-4f41-bbe8-4522f994fb8e\") " pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:50:56.769248 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.769139    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2531468-09bc-4f41-bbe8-4522f994fb8e-dshm\") pod \"rate-test-model-046c73-kserve-54965f5575-cs8d5\" (UID: \"b2531468-09bc-4f41-bbe8-4522f994fb8e\") " pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:50:56.769248 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.769154    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2531468-09bc-4f41-bbe8-4522f994fb8e-model-cache\") pod \"rate-test-model-046c73-kserve-54965f5575-cs8d5\" (UID: \"b2531468-09bc-4f41-bbe8-4522f994fb8e\") " pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:50:56.769248 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.769173    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2531468-09bc-4f41-bbe8-4522f994fb8e-tmp-dir\") pod \"rate-test-model-046c73-kserve-54965f5575-cs8d5\" (UID: \"b2531468-09bc-4f41-bbe8-4522f994fb8e\") " pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:50:56.769248 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.769214    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2531468-09bc-4f41-bbe8-4522f994fb8e-tls-certs\") pod \"rate-test-model-046c73-kserve-54965f5575-cs8d5\" (UID: \"b2531468-09bc-4f41-bbe8-4522f994fb8e\") " pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:50:56.870529 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.870489    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2531468-09bc-4f41-bbe8-4522f994fb8e-home\") pod \"rate-test-model-046c73-kserve-54965f5575-cs8d5\" (UID: \"b2531468-09bc-4f41-bbe8-4522f994fb8e\") " pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:50:56.870529 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.870525    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2531468-09bc-4f41-bbe8-4522f994fb8e-dshm\") pod \"rate-test-model-046c73-kserve-54965f5575-cs8d5\" (UID: \"b2531468-09bc-4f41-bbe8-4522f994fb8e\") " pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:50:56.870751 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.870652    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2531468-09bc-4f41-bbe8-4522f994fb8e-model-cache\") pod \"rate-test-model-046c73-kserve-54965f5575-cs8d5\" (UID: \"b2531468-09bc-4f41-bbe8-4522f994fb8e\") " pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:50:56.870751 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.870711    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2531468-09bc-4f41-bbe8-4522f994fb8e-tmp-dir\") pod \"rate-test-model-046c73-kserve-54965f5575-cs8d5\" (UID: \"b2531468-09bc-4f41-bbe8-4522f994fb8e\") " pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:50:56.870862 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.870755    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2531468-09bc-4f41-bbe8-4522f994fb8e-tls-certs\") pod \"rate-test-model-046c73-kserve-54965f5575-cs8d5\" (UID: \"b2531468-09bc-4f41-bbe8-4522f994fb8e\") " pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:50:56.870963 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.870939    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6865q\" (UniqueName: \"kubernetes.io/projected/b2531468-09bc-4f41-bbe8-4522f994fb8e-kube-api-access-6865q\") pod \"rate-test-model-046c73-kserve-54965f5575-cs8d5\" (UID: \"b2531468-09bc-4f41-bbe8-4522f994fb8e\") " pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:50:56.871128 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.871105    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2531468-09bc-4f41-bbe8-4522f994fb8e-home\") pod \"rate-test-model-046c73-kserve-54965f5575-cs8d5\" (UID: \"b2531468-09bc-4f41-bbe8-4522f994fb8e\") " pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:50:56.871218 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.871153    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2531468-09bc-4f41-bbe8-4522f994fb8e-tmp-dir\") pod \"rate-test-model-046c73-kserve-54965f5575-cs8d5\" (UID: \"b2531468-09bc-4f41-bbe8-4522f994fb8e\") " pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:50:56.871489 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.871468    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2531468-09bc-4f41-bbe8-4522f994fb8e-model-cache\") pod \"rate-test-model-046c73-kserve-54965f5575-cs8d5\" (UID: \"b2531468-09bc-4f41-bbe8-4522f994fb8e\") " pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:50:56.872808 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.872787    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2531468-09bc-4f41-bbe8-4522f994fb8e-dshm\") pod \"rate-test-model-046c73-kserve-54965f5575-cs8d5\" (UID: \"b2531468-09bc-4f41-bbe8-4522f994fb8e\") " pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:50:56.873224 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.873207    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2531468-09bc-4f41-bbe8-4522f994fb8e-tls-certs\") pod \"rate-test-model-046c73-kserve-54965f5575-cs8d5\" (UID: \"b2531468-09bc-4f41-bbe8-4522f994fb8e\") " pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:50:56.880160 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:56.880130    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6865q\" (UniqueName: \"kubernetes.io/projected/b2531468-09bc-4f41-bbe8-4522f994fb8e-kube-api-access-6865q\") pod \"rate-test-model-046c73-kserve-54965f5575-cs8d5\" (UID: \"b2531468-09bc-4f41-bbe8-4522f994fb8e\") " pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:50:57.013060 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:57.012977    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:50:57.146865 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:57.146839    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"]
Jul 20 21:50:57.149064 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:50:57.149039    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb2531468_09bc_4f41_bbe8_4522f994fb8e.slice/crio-9be9836d008b96f7270441794c03616cd18cdb77ac614f5fdf97b4362aad5276 WatchSource:0}: Error finding container 9be9836d008b96f7270441794c03616cd18cdb77ac614f5fdf97b4362aad5276: Status 404 returned error can't find the container with id 9be9836d008b96f7270441794c03616cd18cdb77ac614f5fdf97b4362aad5276
Jul 20 21:50:57.809142 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:57.809102    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5" event={"ID":"b2531468-09bc-4f41-bbe8-4522f994fb8e","Type":"ContainerStarted","Data":"5a2aedad9f2615ef82e40a4a237bff6f6a7e9d8b6a3c07e2a27858fd80c03b1d"}
Jul 20 21:50:57.809142 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:57.809148    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5" event={"ID":"b2531468-09bc-4f41-bbe8-4522f994fb8e","Type":"ContainerStarted","Data":"9be9836d008b96f7270441794c03616cd18cdb77ac614f5fdf97b4362aad5276"}
Jul 20 21:50:57.829257 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:50:57.829202    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5" podStartSLOduration=1.829186795 podStartE2EDuration="1.829186795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:50:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:50:57.827123431 +0000 UTC m=+2898.731113007" watchObservedRunningTime="2026-07-20 21:50:57.829186795 +0000 UTC m=+2898.733176403"
Jul 20 21:51:00.894622 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:00.894583    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:51:07.013331 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:07.013296    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:51:07.013741 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:07.013372    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:51:07.025639 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:07.025612    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:51:07.864670 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:07.864645    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:51:20.098595 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:20.098562    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:51:34.393495 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:34.393461    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:51:41.043967 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:41.043927    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"]
Jul 20 21:51:41.954120 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:41.954068    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"]
Jul 20 21:51:54.136556 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:54.136518    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:51:56.063606 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:56.063563    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk" podUID="d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90" containerName="main" containerID="cri-o://7a3afc5f93c58e826cfe303f1133c1c448d1c9ebab50884672e73ff01ba6e34b" gracePeriod=45
Jul 20 21:51:56.320696 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:56.320633    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:51:56.428667 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:56.428626    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-model-cache\") pod \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\" (UID: \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\") "
Jul 20 21:51:56.428866 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:56.428677    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-home\") pod \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\" (UID: \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\") "
Jul 20 21:51:56.428866 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:56.428716    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dthn\" (UniqueName: \"kubernetes.io/projected/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-kube-api-access-2dthn\") pod \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\" (UID: \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\") "
Jul 20 21:51:56.428866 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:56.428767    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-tmp-dir\") pod \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\" (UID: \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\") "
Jul 20 21:51:56.428866 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:56.428784    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-dshm\") pod \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\" (UID: \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\") "
Jul 20 21:51:56.428866 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:56.428803    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-tls-certs\") pod \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\" (UID: \"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90\") "
Jul 20 21:51:56.429149 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:56.428983    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-model-cache" (OuterVolumeSpecName: "model-cache") pod "d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90" (UID: "d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:51:56.429149 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:56.429007    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-home" (OuterVolumeSpecName: "home") pod "d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90" (UID: "d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:51:56.429149 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:56.429111    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90" (UID: "d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:51:56.429149 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:56.429115    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-model-cache\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:51:56.431042 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:56.431021    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-kube-api-access-2dthn" (OuterVolumeSpecName: "kube-api-access-2dthn") pod "d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90" (UID: "d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90"). InnerVolumeSpecName "kube-api-access-2dthn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 21:51:56.431042 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:56.431035    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90" (UID: "d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 21:51:56.431188 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:56.431112    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-dshm" (OuterVolumeSpecName: "dshm") pod "d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90" (UID: "d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:51:56.530500 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:56.530460    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-home\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:51:56.530500 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:56.530498    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dthn\" (UniqueName: \"kubernetes.io/projected/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-kube-api-access-2dthn\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:51:56.530731 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:56.530512    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-tmp-dir\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:51:56.530731 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:56.530525    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-dshm\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:51:56.530731 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:56.530536    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90-tls-certs\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:51:56.973922 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:56.973855    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5" podUID="b2531468-09bc-4f41-bbe8-4522f994fb8e" containerName="main" containerID="cri-o://5a2aedad9f2615ef82e40a4a237bff6f6a7e9d8b6a3c07e2a27858fd80c03b1d" gracePeriod=45
Jul 20 21:51:57.083582 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.083554    2561 generic.go:356] "Generic (PLEG): container finished" podID="d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90" containerID="7a3afc5f93c58e826cfe303f1133c1c448d1c9ebab50884672e73ff01ba6e34b" exitCode=0
Jul 20 21:51:57.083979 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.083594    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"
Jul 20 21:51:57.083979 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.083601    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk" event={"ID":"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90","Type":"ContainerDied","Data":"7a3afc5f93c58e826cfe303f1133c1c448d1c9ebab50884672e73ff01ba6e34b"}
Jul 20 21:51:57.083979 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.083627    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk" event={"ID":"d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90","Type":"ContainerDied","Data":"86746db3b945088dad7c0b860aacdf3a2c685e899fd16a5de19522df5ef98914"}
Jul 20 21:51:57.083979 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.083644    2561 scope.go:117] "RemoveContainer" containerID="7a3afc5f93c58e826cfe303f1133c1c448d1c9ebab50884672e73ff01ba6e34b"
Jul 20 21:51:57.092515 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.092490    2561 scope.go:117] "RemoveContainer" containerID="7a3afc5f93c58e826cfe303f1133c1c448d1c9ebab50884672e73ff01ba6e34b"
Jul 20 21:51:57.092859 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:51:57.092829    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a3afc5f93c58e826cfe303f1133c1c448d1c9ebab50884672e73ff01ba6e34b\": container with ID starting with 7a3afc5f93c58e826cfe303f1133c1c448d1c9ebab50884672e73ff01ba6e34b not found: ID does not exist" containerID="7a3afc5f93c58e826cfe303f1133c1c448d1c9ebab50884672e73ff01ba6e34b"
Jul 20 21:51:57.092935 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.092876    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a3afc5f93c58e826cfe303f1133c1c448d1c9ebab50884672e73ff01ba6e34b"} err="failed to get container status \"7a3afc5f93c58e826cfe303f1133c1c448d1c9ebab50884672e73ff01ba6e34b\": rpc error: code = NotFound desc = could not find container \"7a3afc5f93c58e826cfe303f1133c1c448d1c9ebab50884672e73ff01ba6e34b\": container with ID starting with 7a3afc5f93c58e826cfe303f1133c1c448d1c9ebab50884672e73ff01ba6e34b not found: ID does not exist"
Jul 20 21:51:57.111869 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.111839    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"]
Jul 20 21:51:57.116556 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.116532    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-196c99/rate-test-model-196c99-kserve-7dbc86ff9f-jpblk"]
Jul 20 21:51:57.228603 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.228544    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:51:57.338402 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.338361    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2531468-09bc-4f41-bbe8-4522f994fb8e-home\") pod \"b2531468-09bc-4f41-bbe8-4522f994fb8e\" (UID: \"b2531468-09bc-4f41-bbe8-4522f994fb8e\") "
Jul 20 21:51:57.338402 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.338409    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6865q\" (UniqueName: \"kubernetes.io/projected/b2531468-09bc-4f41-bbe8-4522f994fb8e-kube-api-access-6865q\") pod \"b2531468-09bc-4f41-bbe8-4522f994fb8e\" (UID: \"b2531468-09bc-4f41-bbe8-4522f994fb8e\") "
Jul 20 21:51:57.338662 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.338442    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2531468-09bc-4f41-bbe8-4522f994fb8e-dshm\") pod \"b2531468-09bc-4f41-bbe8-4522f994fb8e\" (UID: \"b2531468-09bc-4f41-bbe8-4522f994fb8e\") "
Jul 20 21:51:57.338662 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.338458    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2531468-09bc-4f41-bbe8-4522f994fb8e-tmp-dir\") pod \"b2531468-09bc-4f41-bbe8-4522f994fb8e\" (UID: \"b2531468-09bc-4f41-bbe8-4522f994fb8e\") "
Jul 20 21:51:57.338662 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.338473    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2531468-09bc-4f41-bbe8-4522f994fb8e-model-cache\") pod \"b2531468-09bc-4f41-bbe8-4522f994fb8e\" (UID: \"b2531468-09bc-4f41-bbe8-4522f994fb8e\") "
Jul 20 21:51:57.338662 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.338506    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2531468-09bc-4f41-bbe8-4522f994fb8e-tls-certs\") pod \"b2531468-09bc-4f41-bbe8-4522f994fb8e\" (UID: \"b2531468-09bc-4f41-bbe8-4522f994fb8e\") "
Jul 20 21:51:57.338871 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.338705    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2531468-09bc-4f41-bbe8-4522f994fb8e-home" (OuterVolumeSpecName: "home") pod "b2531468-09bc-4f41-bbe8-4522f994fb8e" (UID: "b2531468-09bc-4f41-bbe8-4522f994fb8e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:51:57.338871 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.338784    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2531468-09bc-4f41-bbe8-4522f994fb8e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b2531468-09bc-4f41-bbe8-4522f994fb8e" (UID: "b2531468-09bc-4f41-bbe8-4522f994fb8e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:51:57.338871 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.338794    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2531468-09bc-4f41-bbe8-4522f994fb8e-model-cache" (OuterVolumeSpecName: "model-cache") pod "b2531468-09bc-4f41-bbe8-4522f994fb8e" (UID: "b2531468-09bc-4f41-bbe8-4522f994fb8e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:51:57.340653 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.340634    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2531468-09bc-4f41-bbe8-4522f994fb8e-dshm" (OuterVolumeSpecName: "dshm") pod "b2531468-09bc-4f41-bbe8-4522f994fb8e" (UID: "b2531468-09bc-4f41-bbe8-4522f994fb8e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 21:51:57.340653 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.340640    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2531468-09bc-4f41-bbe8-4522f994fb8e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b2531468-09bc-4f41-bbe8-4522f994fb8e" (UID: "b2531468-09bc-4f41-bbe8-4522f994fb8e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 21:51:57.340760 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.340724    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2531468-09bc-4f41-bbe8-4522f994fb8e-kube-api-access-6865q" (OuterVolumeSpecName: "kube-api-access-6865q") pod "b2531468-09bc-4f41-bbe8-4522f994fb8e" (UID: "b2531468-09bc-4f41-bbe8-4522f994fb8e"). InnerVolumeSpecName "kube-api-access-6865q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 21:51:57.439911 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.439865    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2531468-09bc-4f41-bbe8-4522f994fb8e-home\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:51:57.439911 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.439902    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6865q\" (UniqueName: \"kubernetes.io/projected/b2531468-09bc-4f41-bbe8-4522f994fb8e-kube-api-access-6865q\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:51:57.439911 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.439916    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2531468-09bc-4f41-bbe8-4522f994fb8e-dshm\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:51:57.440192 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.439928    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2531468-09bc-4f41-bbe8-4522f994fb8e-tmp-dir\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:51:57.440192 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.439939    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2531468-09bc-4f41-bbe8-4522f994fb8e-model-cache\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:51:57.440192 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.439949    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2531468-09bc-4f41-bbe8-4522f994fb8e-tls-certs\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:51:57.725635 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:57.725601    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90" path="/var/lib/kubelet/pods/d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90/volumes"
Jul 20 21:51:58.089922 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:58.089886    2561 generic.go:356] "Generic (PLEG): container finished" podID="b2531468-09bc-4f41-bbe8-4522f994fb8e" containerID="5a2aedad9f2615ef82e40a4a237bff6f6a7e9d8b6a3c07e2a27858fd80c03b1d" exitCode=0
Jul 20 21:51:58.089922 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:58.089928    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"
Jul 20 21:51:58.090465 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:58.089977    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5" event={"ID":"b2531468-09bc-4f41-bbe8-4522f994fb8e","Type":"ContainerDied","Data":"5a2aedad9f2615ef82e40a4a237bff6f6a7e9d8b6a3c07e2a27858fd80c03b1d"}
Jul 20 21:51:58.090465 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:58.090017    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5" event={"ID":"b2531468-09bc-4f41-bbe8-4522f994fb8e","Type":"ContainerDied","Data":"9be9836d008b96f7270441794c03616cd18cdb77ac614f5fdf97b4362aad5276"}
Jul 20 21:51:58.090465 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:58.090036    2561 scope.go:117] "RemoveContainer" containerID="5a2aedad9f2615ef82e40a4a237bff6f6a7e9d8b6a3c07e2a27858fd80c03b1d"
Jul 20 21:51:58.098797 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:58.098777    2561 scope.go:117] "RemoveContainer" containerID="5a2aedad9f2615ef82e40a4a237bff6f6a7e9d8b6a3c07e2a27858fd80c03b1d"
Jul 20 21:51:58.099112 ip-10-0-128-253 kubenswrapper[2561]: E0720 21:51:58.099090    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a2aedad9f2615ef82e40a4a237bff6f6a7e9d8b6a3c07e2a27858fd80c03b1d\": container with ID starting with 5a2aedad9f2615ef82e40a4a237bff6f6a7e9d8b6a3c07e2a27858fd80c03b1d not found: ID does not exist" containerID="5a2aedad9f2615ef82e40a4a237bff6f6a7e9d8b6a3c07e2a27858fd80c03b1d"
Jul 20 21:51:58.099216 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:58.099119    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a2aedad9f2615ef82e40a4a237bff6f6a7e9d8b6a3c07e2a27858fd80c03b1d"} err="failed to get container status \"5a2aedad9f2615ef82e40a4a237bff6f6a7e9d8b6a3c07e2a27858fd80c03b1d\": rpc error: code = NotFound desc = could not find container \"5a2aedad9f2615ef82e40a4a237bff6f6a7e9d8b6a3c07e2a27858fd80c03b1d\": container with ID starting with 5a2aedad9f2615ef82e40a4a237bff6f6a7e9d8b6a3c07e2a27858fd80c03b1d not found: ID does not exist"
Jul 20 21:51:58.114192 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:58.114166    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"]
Jul 20 21:51:58.122563 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:58.122539    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-046c73/rate-test-model-046c73-kserve-54965f5575-cs8d5"]
Jul 20 21:51:58.431289 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:58.431202    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7634-140077526717472"]
Jul 20 21:51:58.431540 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:58.431528    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90" containerName="main"
Jul 20 21:51:58.431595 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:58.431542    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90" containerName="main"
Jul 20 21:51:58.431595 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:58.431549    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2531468-09bc-4f41-bbe8-4522f994fb8e" containerName="main"
Jul 20 21:51:58.431595 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:58.431555    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2531468-09bc-4f41-bbe8-4522f994fb8e" containerName="main"
Jul 20 21:51:58.431701 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:58.431618    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4fd8e36-71c8-4c57-a3b7-5ed91ed11a90" containerName="main"
Jul 20 21:51:58.431701 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:58.431627    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2531468-09bc-4f41-bbe8-4522f994fb8e" containerName="main"
Jul 20 21:51:58.441492 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:58.441465    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7634-140077526717472"
Jul 20 21:51:58.445050 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:58.445023    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lhmnw\""
Jul 20 21:51:58.445193 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:58.445159    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7634-140077526717472"]
Jul 20 21:51:58.549000 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:58.548962    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tdvm\" (UniqueName: \"kubernetes.io/projected/c7072db5-a2c4-4385-91c5-ed6225240e3c-kube-api-access-4tdvm\") pod \"test-curl-7634-140077526717472\" (UID: \"c7072db5-a2c4-4385-91c5-ed6225240e3c\") " pod="opendatahub/test-curl-7634-140077526717472"
Jul 20 21:51:58.649748 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:58.649709    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tdvm\" (UniqueName: \"kubernetes.io/projected/c7072db5-a2c4-4385-91c5-ed6225240e3c-kube-api-access-4tdvm\") pod \"test-curl-7634-140077526717472\" (UID: \"c7072db5-a2c4-4385-91c5-ed6225240e3c\") " pod="opendatahub/test-curl-7634-140077526717472"
Jul 20 21:51:58.659339 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:58.659312    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tdvm\" (UniqueName: \"kubernetes.io/projected/c7072db5-a2c4-4385-91c5-ed6225240e3c-kube-api-access-4tdvm\") pod \"test-curl-7634-140077526717472\" (UID: \"c7072db5-a2c4-4385-91c5-ed6225240e3c\") " pod="opendatahub/test-curl-7634-140077526717472"
Jul 20 21:51:58.751947 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:58.751853    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7634-140077526717472"
Jul 20 21:51:59.093313 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:59.092871    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7634-140077526717472"]
Jul 20 21:51:59.095845 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:51:59.095812    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc7072db5_a2c4_4385_91c5_ed6225240e3c.slice/crio-ef773dac6383dea1893a0fb4579d9353bc1d56e6533a4b3e0d60afc91269e00b WatchSource:0}: Error finding container ef773dac6383dea1893a0fb4579d9353bc1d56e6533a4b3e0d60afc91269e00b: Status 404 returned error can't find the container with id ef773dac6383dea1893a0fb4579d9353bc1d56e6533a4b3e0d60afc91269e00b
Jul 20 21:51:59.097129 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:59.097111    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 21:51:59.726782 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:51:59.726749    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2531468-09bc-4f41-bbe8-4522f994fb8e" path="/var/lib/kubelet/pods/b2531468-09bc-4f41-bbe8-4522f994fb8e/volumes"
Jul 20 21:52:00.102302 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:00.102266    2561 generic.go:356] "Generic (PLEG): container finished" podID="c7072db5-a2c4-4385-91c5-ed6225240e3c" containerID="2b795a5c1050bde6dae73122334068430e366bbe62c55edc27a9ba1bcb09bea5" exitCode=0
Jul 20 21:52:00.102682 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:00.102347    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7634-140077526717472" event={"ID":"c7072db5-a2c4-4385-91c5-ed6225240e3c","Type":"ContainerDied","Data":"2b795a5c1050bde6dae73122334068430e366bbe62c55edc27a9ba1bcb09bea5"}
Jul 20 21:52:00.102682 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:00.102394    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7634-140077526717472" event={"ID":"c7072db5-a2c4-4385-91c5-ed6225240e3c","Type":"ContainerStarted","Data":"ef773dac6383dea1893a0fb4579d9353bc1d56e6533a4b3e0d60afc91269e00b"}
Jul 20 21:52:01.244176 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:01.244150    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7634-140077526717472"
Jul 20 21:52:01.274056 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:01.274028    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tdvm\" (UniqueName: \"kubernetes.io/projected/c7072db5-a2c4-4385-91c5-ed6225240e3c-kube-api-access-4tdvm\") pod \"c7072db5-a2c4-4385-91c5-ed6225240e3c\" (UID: \"c7072db5-a2c4-4385-91c5-ed6225240e3c\") "
Jul 20 21:52:01.276195 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:01.276170    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7072db5-a2c4-4385-91c5-ed6225240e3c-kube-api-access-4tdvm" (OuterVolumeSpecName: "kube-api-access-4tdvm") pod "c7072db5-a2c4-4385-91c5-ed6225240e3c" (UID: "c7072db5-a2c4-4385-91c5-ed6225240e3c"). InnerVolumeSpecName "kube-api-access-4tdvm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 21:52:01.289344 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:01.289323    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7634-140077526717472_c7072db5-a2c4-4385-91c5-ed6225240e3c/test-curl-7634-140077526717472/0.log"
Jul 20 21:52:01.318350 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:01.318312    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7634-140077526717472"]
Jul 20 21:52:01.324338 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:01.324315    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7634-140077526717472"]
Jul 20 21:52:01.374906 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:01.374833    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tdvm\" (UniqueName: \"kubernetes.io/projected/c7072db5-a2c4-4385-91c5-ed6225240e3c-kube-api-access-4tdvm\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:52:01.526758 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:01.526725    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196c99-b7tq6"]
Jul 20 21:52:01.527110 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:01.527096    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7072db5-a2c4-4385-91c5-ed6225240e3c" containerName="test-curl-7634-140077526717472"
Jul 20 21:52:01.527160 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:01.527112    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7072db5-a2c4-4385-91c5-ed6225240e3c" containerName="test-curl-7634-140077526717472"
Jul 20 21:52:01.527201 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:01.527171    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7072db5-a2c4-4385-91c5-ed6225240e3c" containerName="test-curl-7634-140077526717472"
Jul 20 21:52:01.530031 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:01.530016    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196c99-b7tq6"
Jul 20 21:52:01.533120 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:01.533100    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 21:52:01.533399 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:01.533386    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qr6f9\""
Jul 20 21:52:01.541002 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:01.540976    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196c99-b7tq6"]
Jul 20 21:52:01.575803 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:01.575761    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a497c16-c8b4-4623-b0e1-802afa657be9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-196c99-b7tq6\" (UID: \"9a497c16-c8b4-4623-b0e1-802afa657be9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196c99-b7tq6"
Jul 20 21:52:01.677191 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:01.677092    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a497c16-c8b4-4623-b0e1-802afa657be9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-196c99-b7tq6\" (UID: \"9a497c16-c8b4-4623-b0e1-802afa657be9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196c99-b7tq6"
Jul 20 21:52:01.677694 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:01.677674    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a497c16-c8b4-4623-b0e1-802afa657be9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-196c99-b7tq6\" (UID: \"9a497c16-c8b4-4623-b0e1-802afa657be9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196c99-b7tq6"
Jul 20 21:52:01.725653 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:01.725622    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7072db5-a2c4-4385-91c5-ed6225240e3c" path="/var/lib/kubelet/pods/c7072db5-a2c4-4385-91c5-ed6225240e3c/volumes"
Jul 20 21:52:01.841096 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:01.841031    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196c99-b7tq6"
Jul 20 21:52:01.986607 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:01.986582    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196c99-b7tq6"]
Jul 20 21:52:01.988928 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:52:01.988899    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a497c16_c8b4_4623_b0e1_802afa657be9.slice/crio-bcb2e14bb41acc1a270c3fea8ead19c5618bd545da6998a62cb65f07e700b004 WatchSource:0}: Error finding container bcb2e14bb41acc1a270c3fea8ead19c5618bd545da6998a62cb65f07e700b004: Status 404 returned error can't find the container with id bcb2e14bb41acc1a270c3fea8ead19c5618bd545da6998a62cb65f07e700b004
Jul 20 21:52:02.113788 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:02.113751    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196c99-b7tq6" event={"ID":"9a497c16-c8b4-4623-b0e1-802afa657be9","Type":"ContainerStarted","Data":"bcb2e14bb41acc1a270c3fea8ead19c5618bd545da6998a62cb65f07e700b004"}
Jul 20 21:52:02.114877 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:02.114852    2561 scope.go:117] "RemoveContainer" containerID="2b795a5c1050bde6dae73122334068430e366bbe62c55edc27a9ba1bcb09bea5"
Jul 20 21:52:02.115017 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:02.114881    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7634-140077526717472"
Jul 20 21:52:03.120945 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:03.120911    2561 generic.go:356] "Generic (PLEG): container finished" podID="9a497c16-c8b4-4623-b0e1-802afa657be9" containerID="91eabfa2d8567d4cf197d0919797c617bfd4a14a710d77da5a4c1f9529a0b7c4" exitCode=0
Jul 20 21:52:03.121400 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:03.120994    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196c99-b7tq6" event={"ID":"9a497c16-c8b4-4623-b0e1-802afa657be9","Type":"ContainerDied","Data":"91eabfa2d8567d4cf197d0919797c617bfd4a14a710d77da5a4c1f9529a0b7c4"}
Jul 20 21:52:04.250554 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:04.250530    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196c99-b7tq6"
Jul 20 21:52:04.301696 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:04.301669    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a497c16-c8b4-4623-b0e1-802afa657be9-maas-api-service-ca\") pod \"9a497c16-c8b4-4623-b0e1-802afa657be9\" (UID: \"9a497c16-c8b4-4623-b0e1-802afa657be9\") "
Jul 20 21:52:04.302019 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:04.301996    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a497c16-c8b4-4623-b0e1-802afa657be9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9a497c16-c8b4-4623-b0e1-802afa657be9" (UID: "9a497c16-c8b4-4623-b0e1-802afa657be9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 21:52:04.402463 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:04.402358    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a497c16-c8b4-4623-b0e1-802afa657be9-maas-api-service-ca\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:52:05.131791 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:05.131764    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196c99-b7tq6"
Jul 20 21:52:05.131969 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:05.131763    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196c99-b7tq6" event={"ID":"9a497c16-c8b4-4623-b0e1-802afa657be9","Type":"ContainerDied","Data":"bcb2e14bb41acc1a270c3fea8ead19c5618bd545da6998a62cb65f07e700b004"}
Jul 20 21:52:05.131969 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:05.131893    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bcb2e14bb41acc1a270c3fea8ead19c5618bd545da6998a62cb65f07e700b004"
Jul 20 21:52:14.905587 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:14.905549    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:52:16.653906 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:16.653870    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196c99-b7tq6"]
Jul 20 21:52:16.657101 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:16.657047    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-196c99-b7tq6"]
Jul 20 21:52:17.725667 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:17.725633    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a497c16-c8b4-4623-b0e1-802afa657be9" path="/var/lib/kubelet/pods/9a497c16-c8b4-4623-b0e1-802afa657be9/volumes"
Jul 20 21:52:25.120297 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:25.120258    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-046c73-hppxc"]
Jul 20 21:52:25.120673 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:25.120593    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a497c16-c8b4-4623-b0e1-802afa657be9" containerName="revoke-keys"
Jul 20 21:52:25.120673 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:25.120604    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a497c16-c8b4-4623-b0e1-802afa657be9" containerName="revoke-keys"
Jul 20 21:52:25.120673 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:25.120663    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a497c16-c8b4-4623-b0e1-802afa657be9" containerName="revoke-keys"
Jul 20 21:52:25.123656 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:25.123639    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-046c73-hppxc"
Jul 20 21:52:25.126556 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:25.126536    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qr6f9\""
Jul 20 21:52:25.127289 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:25.127273    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 21:52:25.139656 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:25.139626    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-046c73-hppxc"]
Jul 20 21:52:25.291781 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:25.291739    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/150883b6-e0c5-4003-bd7f-eeb976756f39-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-046c73-hppxc\" (UID: \"150883b6-e0c5-4003-bd7f-eeb976756f39\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-046c73-hppxc"
Jul 20 21:52:25.392786 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:25.392670    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/150883b6-e0c5-4003-bd7f-eeb976756f39-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-046c73-hppxc\" (UID: \"150883b6-e0c5-4003-bd7f-eeb976756f39\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-046c73-hppxc"
Jul 20 21:52:25.393446 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:25.393421    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/150883b6-e0c5-4003-bd7f-eeb976756f39-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-046c73-hppxc\" (UID: \"150883b6-e0c5-4003-bd7f-eeb976756f39\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-046c73-hppxc"
Jul 20 21:52:25.433265 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:25.433233    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-046c73-hppxc"
Jul 20 21:52:25.574113 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:25.574087    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-046c73-hppxc"]
Jul 20 21:52:25.576555 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:52:25.576524    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod150883b6_e0c5_4003_bd7f_eeb976756f39.slice/crio-bb8196c1eff97dd8b7e9421d0a99aa1f30a355a965db0c610b6b8345bf6c5314 WatchSource:0}: Error finding container bb8196c1eff97dd8b7e9421d0a99aa1f30a355a965db0c610b6b8345bf6c5314: Status 404 returned error can't find the container with id bb8196c1eff97dd8b7e9421d0a99aa1f30a355a965db0c610b6b8345bf6c5314
Jul 20 21:52:26.231046 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:26.231006    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-046c73-hppxc" event={"ID":"150883b6-e0c5-4003-bd7f-eeb976756f39","Type":"ContainerStarted","Data":"eaee40f29dea96b99107c01d8e822c584467e8974d3346203c32ffef6422f533"}
Jul 20 21:52:26.231457 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:26.231107    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-046c73-hppxc" event={"ID":"150883b6-e0c5-4003-bd7f-eeb976756f39","Type":"ContainerStarted","Data":"bb8196c1eff97dd8b7e9421d0a99aa1f30a355a965db0c610b6b8345bf6c5314"}
Jul 20 21:52:26.256012 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:26.255958    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-046c73-hppxc" podStartSLOduration=1.255942416 podStartE2EDuration="1.255942416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 21:52:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:52:26.254553531 +0000 UTC m=+2987.158543110" watchObservedRunningTime="2026-07-20 21:52:26.255942416 +0000 UTC m=+2987.159932031"
Jul 20 21:52:27.236791 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:27.236756    2561 generic.go:356] "Generic (PLEG): container finished" podID="150883b6-e0c5-4003-bd7f-eeb976756f39" containerID="eaee40f29dea96b99107c01d8e822c584467e8974d3346203c32ffef6422f533" exitCode=0
Jul 20 21:52:27.237206 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:27.236810    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-046c73-hppxc" event={"ID":"150883b6-e0c5-4003-bd7f-eeb976756f39","Type":"ContainerDied","Data":"eaee40f29dea96b99107c01d8e822c584467e8974d3346203c32ffef6422f533"}
Jul 20 21:52:28.373285 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:28.373259    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-046c73-hppxc"
Jul 20 21:52:28.521596 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:28.521492    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/150883b6-e0c5-4003-bd7f-eeb976756f39-maas-api-service-ca\") pod \"150883b6-e0c5-4003-bd7f-eeb976756f39\" (UID: \"150883b6-e0c5-4003-bd7f-eeb976756f39\") "
Jul 20 21:52:28.521863 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:28.521838    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/150883b6-e0c5-4003-bd7f-eeb976756f39-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "150883b6-e0c5-4003-bd7f-eeb976756f39" (UID: "150883b6-e0c5-4003-bd7f-eeb976756f39"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 21:52:28.622349 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:28.622299    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/150883b6-e0c5-4003-bd7f-eeb976756f39-maas-api-service-ca\") on node \"ip-10-0-128-253.ec2.internal\" DevicePath \"\""
Jul 20 21:52:29.247631 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:29.247598    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-046c73-hppxc"
Jul 20 21:52:29.247631 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:29.247612    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-046c73-hppxc" event={"ID":"150883b6-e0c5-4003-bd7f-eeb976756f39","Type":"ContainerDied","Data":"bb8196c1eff97dd8b7e9421d0a99aa1f30a355a965db0c610b6b8345bf6c5314"}
Jul 20 21:52:29.247848 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:29.247653    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb8196c1eff97dd8b7e9421d0a99aa1f30a355a965db0c610b6b8345bf6c5314"
Jul 20 21:52:38.936177 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:38.936142    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9cb2q"]
Jul 20 21:52:40.277702 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:40.277608    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-046c73-hppxc"]
Jul 20 21:52:40.283068 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:40.283035    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-046c73-hppxc"]
Jul 20 21:52:40.570640 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:40.570609    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/1.log"
Jul 20 21:52:40.579188 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:40.579153    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/1.log"
Jul 20 21:52:41.725426 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:41.725390    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="150883b6-e0c5-4003-bd7f-eeb976756f39" path="/var/lib/kubelet/pods/150883b6-e0c5-4003-bd7f-eeb976756f39/volumes"
Jul 20 21:52:53.941682 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:53.941650    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66b5c4b596-jc2h8_e7b15cf5-17da-4dc2-ab69-25169d4a70bb/copy-manifests/0.log"
Jul 20 21:52:53.962162 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:53.962135    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66b5c4b596-jc2h8_e7b15cf5-17da-4dc2-ab69-25169d4a70bb/manager/0.log"
Jul 20 21:52:54.581991 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:54.581953    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29743050-67qjh_1045438e-4f31-4a0d-a478-30e35b460321/cleanup/0.log"
Jul 20 21:52:56.826235 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:56.826193    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-9cb2q_4b486af1-6a3f-41c8-8ba9-b428fd7abfcb/limitador/0.log"
Jul 20 21:52:57.325828 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:57.325784    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4c9892_f0fb0c79-9936-407c-a54e-ee4dc5cabc03/istio-proxy/0.log"
Jul 20 21:52:57.440464 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:57.440438    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-jp58g_70b96039-dacf-4054-8ffb-e9f20695be71/discovery/0.log"
Jul 20 21:52:58.236883 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:58.236853    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64d7f6dfcb-zmq82_e3fb0dff-5b39-431b-959b-ba6636770fe4/router/0.log"
Jul 20 21:52:59.072845 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:52:59.072812    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-lfktv_7dbbd3a4-752c-4a19-bf00-5fa435dc4258/main/0.log"
Jul 20 21:53:07.499609 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:07.499576    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-g6ctc_6c684495-6967-47c7-ba35-0b9b37110406/global-pull-secret-syncer/0.log"
Jul 20 21:53:07.608640 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:07.608608    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bpvws_bf582cd6-b53c-45c1-b06b-8eea95f85a95/konnectivity-agent/0.log"
Jul 20 21:53:07.664200 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:07.664169    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-253.ec2.internal_daa60d7c62b18ac55a74c9a2a524cd8f/haproxy/0.log"
Jul 20 21:53:13.030460 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:13.030420    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-9cb2q_4b486af1-6a3f-41c8-8ba9-b428fd7abfcb/limitador/0.log"
Jul 20 21:53:14.739518 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:14.739483    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dab17d8e-c490-4e18-bf39-b44cb28eb8db/alertmanager/0.log"
Jul 20 21:53:14.794873 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:14.794848    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dab17d8e-c490-4e18-bf39-b44cb28eb8db/config-reloader/0.log"
Jul 20 21:53:14.850316 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:14.850282    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dab17d8e-c490-4e18-bf39-b44cb28eb8db/kube-rbac-proxy-web/0.log"
Jul 20 21:53:14.888645 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:14.888621    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dab17d8e-c490-4e18-bf39-b44cb28eb8db/kube-rbac-proxy/0.log"
Jul 20 21:53:14.921191 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:14.921167    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dab17d8e-c490-4e18-bf39-b44cb28eb8db/kube-rbac-proxy-metric/0.log"
Jul 20 21:53:14.955608 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:14.955579    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dab17d8e-c490-4e18-bf39-b44cb28eb8db/prom-label-proxy/0.log"
Jul 20 21:53:15.000898 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:15.000815    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dab17d8e-c490-4e18-bf39-b44cb28eb8db/init-config-reloader/0.log"
Jul 20 21:53:15.322037 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:15.321965    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4csfn_83bc0281-5b82-42cb-92a6-3652a24925cc/node-exporter/0.log"
Jul 20 21:53:15.370735 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:15.370709    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4csfn_83bc0281-5b82-42cb-92a6-3652a24925cc/kube-rbac-proxy/0.log"
Jul 20 21:53:15.409157 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:15.409135    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4csfn_83bc0281-5b82-42cb-92a6-3652a24925cc/init-textfile/0.log"
Jul 20 21:53:15.701700 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:15.701621    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-kdnkm_22e52d64-9dd4-4aa4-9b40-c3c3b2230975/kube-rbac-proxy-main/0.log"
Jul 20 21:53:15.758219 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:15.758190    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-kdnkm_22e52d64-9dd4-4aa4-9b40-c3c3b2230975/kube-rbac-proxy-self/0.log"
Jul 20 21:53:15.822106 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:15.822059    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-kdnkm_22e52d64-9dd4-4aa4-9b40-c3c3b2230975/openshift-state-metrics/0.log"
Jul 20 21:53:16.107627 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:16.107600    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-6t8bd_c71af73b-6f46-46a3-b6ca-f1ddb5d4f240/prometheus-operator/0.log"
Jul 20 21:53:16.137682 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:16.137658    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-6t8bd_c71af73b-6f46-46a3-b6ca-f1ddb5d4f240/kube-rbac-proxy/0.log"
Jul 20 21:53:17.014010 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.013960    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cffm9/perf-node-gather-daemonset-4kj9p"]
Jul 20 21:53:17.014509 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.014468    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="150883b6-e0c5-4003-bd7f-eeb976756f39" containerName="revoke-keys"
Jul 20 21:53:17.014509 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.014486    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="150883b6-e0c5-4003-bd7f-eeb976756f39" containerName="revoke-keys"
Jul 20 21:53:17.014631 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.014602    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="150883b6-e0c5-4003-bd7f-eeb976756f39" containerName="revoke-keys"
Jul 20 21:53:17.017571 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.017550    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cffm9/perf-node-gather-daemonset-4kj9p"
Jul 20 21:53:17.020786 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.020766    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cffm9\"/\"kube-root-ca.crt\""
Jul 20 21:53:17.021643 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.021627    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-cffm9\"/\"default-dockercfg-tbtzm\""
Jul 20 21:53:17.021702 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.021649    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cffm9\"/\"openshift-service-ca.crt\""
Jul 20 21:53:17.034102 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.034063    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cffm9/perf-node-gather-daemonset-4kj9p"]
Jul 20 21:53:17.159777 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.159746    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e270d217-efd1-400f-a2d3-8f358fed9fe5-sys\") pod \"perf-node-gather-daemonset-4kj9p\" (UID: \"e270d217-efd1-400f-a2d3-8f358fed9fe5\") " pod="openshift-must-gather-cffm9/perf-node-gather-daemonset-4kj9p"
Jul 20 21:53:17.159965 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.159793    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e270d217-efd1-400f-a2d3-8f358fed9fe5-podres\") pod \"perf-node-gather-daemonset-4kj9p\" (UID: \"e270d217-efd1-400f-a2d3-8f358fed9fe5\") " pod="openshift-must-gather-cffm9/perf-node-gather-daemonset-4kj9p"
Jul 20 21:53:17.159965 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.159820    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e270d217-efd1-400f-a2d3-8f358fed9fe5-lib-modules\") pod \"perf-node-gather-daemonset-4kj9p\" (UID: \"e270d217-efd1-400f-a2d3-8f358fed9fe5\") " pod="openshift-must-gather-cffm9/perf-node-gather-daemonset-4kj9p"
Jul 20 21:53:17.160101 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.159956    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e270d217-efd1-400f-a2d3-8f358fed9fe5-proc\") pod \"perf-node-gather-daemonset-4kj9p\" (UID: \"e270d217-efd1-400f-a2d3-8f358fed9fe5\") " pod="openshift-must-gather-cffm9/perf-node-gather-daemonset-4kj9p"
Jul 20 21:53:17.160101 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.159992    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9qgj\" (UniqueName: \"kubernetes.io/projected/e270d217-efd1-400f-a2d3-8f358fed9fe5-kube-api-access-f9qgj\") pod \"perf-node-gather-daemonset-4kj9p\" (UID: \"e270d217-efd1-400f-a2d3-8f358fed9fe5\") " pod="openshift-must-gather-cffm9/perf-node-gather-daemonset-4kj9p"
Jul 20 21:53:17.261168 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.261126    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e270d217-efd1-400f-a2d3-8f358fed9fe5-proc\") pod \"perf-node-gather-daemonset-4kj9p\" (UID: \"e270d217-efd1-400f-a2d3-8f358fed9fe5\") " pod="openshift-must-gather-cffm9/perf-node-gather-daemonset-4kj9p"
Jul 20 21:53:17.261168 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.261166    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9qgj\" (UniqueName: \"kubernetes.io/projected/e270d217-efd1-400f-a2d3-8f358fed9fe5-kube-api-access-f9qgj\") pod \"perf-node-gather-daemonset-4kj9p\" (UID: \"e270d217-efd1-400f-a2d3-8f358fed9fe5\") " pod="openshift-must-gather-cffm9/perf-node-gather-daemonset-4kj9p"
Jul 20 21:53:17.261410 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.261186    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e270d217-efd1-400f-a2d3-8f358fed9fe5-sys\") pod \"perf-node-gather-daemonset-4kj9p\" (UID: \"e270d217-efd1-400f-a2d3-8f358fed9fe5\") " pod="openshift-must-gather-cffm9/perf-node-gather-daemonset-4kj9p"
Jul 20 21:53:17.261410 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.261221    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e270d217-efd1-400f-a2d3-8f358fed9fe5-podres\") pod \"perf-node-gather-daemonset-4kj9p\" (UID: \"e270d217-efd1-400f-a2d3-8f358fed9fe5\") " pod="openshift-must-gather-cffm9/perf-node-gather-daemonset-4kj9p"
Jul 20 21:53:17.261410 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.261246    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e270d217-efd1-400f-a2d3-8f358fed9fe5-lib-modules\") pod \"perf-node-gather-daemonset-4kj9p\" (UID: \"e270d217-efd1-400f-a2d3-8f358fed9fe5\") " pod="openshift-must-gather-cffm9/perf-node-gather-daemonset-4kj9p"
Jul 20 21:53:17.261410 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.261275    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e270d217-efd1-400f-a2d3-8f358fed9fe5-proc\") pod \"perf-node-gather-daemonset-4kj9p\" (UID: \"e270d217-efd1-400f-a2d3-8f358fed9fe5\") " pod="openshift-must-gather-cffm9/perf-node-gather-daemonset-4kj9p"
Jul 20 21:53:17.261410 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.261332    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e270d217-efd1-400f-a2d3-8f358fed9fe5-sys\") pod \"perf-node-gather-daemonset-4kj9p\" (UID: \"e270d217-efd1-400f-a2d3-8f358fed9fe5\") " pod="openshift-must-gather-cffm9/perf-node-gather-daemonset-4kj9p"
Jul 20 21:53:17.261410 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.261399    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e270d217-efd1-400f-a2d3-8f358fed9fe5-lib-modules\") pod \"perf-node-gather-daemonset-4kj9p\" (UID: \"e270d217-efd1-400f-a2d3-8f358fed9fe5\") " pod="openshift-must-gather-cffm9/perf-node-gather-daemonset-4kj9p"
Jul 20 21:53:17.261608 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.261399    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e270d217-efd1-400f-a2d3-8f358fed9fe5-podres\") pod \"perf-node-gather-daemonset-4kj9p\" (UID: \"e270d217-efd1-400f-a2d3-8f358fed9fe5\") " pod="openshift-must-gather-cffm9/perf-node-gather-daemonset-4kj9p"
Jul 20 21:53:17.270594 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.270534    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9qgj\" (UniqueName: \"kubernetes.io/projected/e270d217-efd1-400f-a2d3-8f358fed9fe5-kube-api-access-f9qgj\") pod \"perf-node-gather-daemonset-4kj9p\" (UID: \"e270d217-efd1-400f-a2d3-8f358fed9fe5\") " pod="openshift-must-gather-cffm9/perf-node-gather-daemonset-4kj9p"
Jul 20 21:53:17.327013 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.326978    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cffm9/perf-node-gather-daemonset-4kj9p"
Jul 20 21:53:17.464472 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:17.464446    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cffm9/perf-node-gather-daemonset-4kj9p"]
Jul 20 21:53:17.467065 ip-10-0-128-253 kubenswrapper[2561]: W0720 21:53:17.467035    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode270d217_efd1_400f_a2d3_8f358fed9fe5.slice/crio-b9c23c6ed582d3a60d7b41245f2c473348dfdb26239a9de9ea2381b86f2c1d83 WatchSource:0}: Error finding container b9c23c6ed582d3a60d7b41245f2c473348dfdb26239a9de9ea2381b86f2c1d83: Status 404 returned error can't find the container with id b9c23c6ed582d3a60d7b41245f2c473348dfdb26239a9de9ea2381b86f2c1d83
Jul 20 21:53:18.137065 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:18.137029    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/1.log"
Jul 20 21:53:18.140960 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:18.140940    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-fz7g7_f3754f5e-54ec-4d60-9e1f-9290b4a82d07/console-operator/2.log"
Jul 20 21:53:18.476794 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:18.476710    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cffm9/perf-node-gather-daemonset-4kj9p" event={"ID":"e270d217-efd1-400f-a2d3-8f358fed9fe5","Type":"ContainerStarted","Data":"1de53456c5d5704f0a8050b14c6ab4c19d5905c325a30816d5f9053c34ee478d"}
Jul 20 21:53:18.476794 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:18.476762    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cffm9/perf-node-gather-daemonset-4kj9p" event={"ID":"e270d217-efd1-400f-a2d3-8f358fed9fe5","Type":"ContainerStarted","Data":"b9c23c6ed582d3a60d7b41245f2c473348dfdb26239a9de9ea2381b86f2c1d83"}
Jul 20 21:53:18.476976 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:18.476837    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-cffm9/perf-node-gather-daemonset-4kj9p"
Jul 20 21:53:18.494850 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:18.494796    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cffm9/perf-node-gather-daemonset-4kj9p" podStartSLOduration=2.494780376 podStartE2EDuration="2.494780376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 21:53:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 21:53:18.493634416 +0000 UTC m=+3039.397623996" watchObservedRunningTime="2026-07-20 21:53:18.494780376 +0000 UTC m=+3039.398769954"
Jul 20 21:53:18.625964 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:18.625936    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-ghswp_ccb3a0dc-353e-4850-9ce2-ddd5bf113e38/download-server/0.log"
Jul 20 21:53:20.288744 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:20.288716    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-prms6_8e2b83ca-b2c3-4899-92ed-16890097cd04/dns/0.log"
Jul 20 21:53:20.310511 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:20.310483    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-prms6_8e2b83ca-b2c3-4899-92ed-16890097cd04/kube-rbac-proxy/0.log"
Jul 20 21:53:20.378763 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:20.378730    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7k6k7_bcabbaa9-9e52-49f0-8786-ae54a37210e3/dns-node-resolver/0.log"
Jul 20 21:53:20.952702 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:20.952654    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7b76f5dbfb-pc4k2_c470afc9-11d3-43c5-b6fb-d599f7e34043/registry/0.log"
Jul 20 21:53:20.983720 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:20.983695    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8zfld_58fafef1-89fd-4094-9993-70ab07fecd85/node-ca/0.log"
Jul 20 21:53:22.025696 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:22.025671    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4c9892_f0fb0c79-9936-407c-a54e-ee4dc5cabc03/istio-proxy/0.log"
Jul 20 21:53:22.155107 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:22.155055    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-jp58g_70b96039-dacf-4054-8ffb-e9f20695be71/discovery/0.log"
Jul 20 21:53:22.402600 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:22.402567    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64d7f6dfcb-zmq82_e3fb0dff-5b39-431b-959b-ba6636770fe4/router/0.log"
Jul 20 21:53:23.106478 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:23.106445    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9g6f9_32060f49-8c0d-425a-a22f-8f032ddf9dea/serve-healthcheck-canary/0.log"
Jul 20 21:53:23.633364 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:23.633332    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-ckxwr_844a855d-9faf-4af8-96f4-1985596720ea/insights-operator/0.log"
Jul 20 21:53:23.633712 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:23.633698    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-ckxwr_844a855d-9faf-4af8-96f4-1985596720ea/insights-operator/1.log"
Jul 20 21:53:23.654574 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:23.654551    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hpljx_7215e114-f9ed-40cb-9de5-c48cef3995cd/kube-rbac-proxy/0.log"
Jul 20 21:53:23.675844 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:23.675824    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hpljx_7215e114-f9ed-40cb-9de5-c48cef3995cd/exporter/0.log"
Jul 20 21:53:23.696222 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:23.696198    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hpljx_7215e114-f9ed-40cb-9de5-c48cef3995cd/extractor/0.log"
Jul 20 21:53:24.491085 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:24.491047    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-cffm9/perf-node-gather-daemonset-4kj9p"
Jul 20 21:53:26.882998 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:26.882966    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66b5c4b596-jc2h8_e7b15cf5-17da-4dc2-ab69-25169d4a70bb/manager/0.log"
Jul 20 21:53:26.904231 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:26.904204    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66b5c4b596-jc2h8_e7b15cf5-17da-4dc2-ab69-25169d4a70bb/copy-manifests/0.log"
Jul 20 21:53:27.340611 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:27.340579    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4c9892_f0fb0c79-9936-407c-a54e-ee4dc5cabc03/istio-proxy/0.log"
Jul 20 21:53:27.453197 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:27.453164    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-jp58g_70b96039-dacf-4054-8ffb-e9f20695be71/discovery/0.log"
Jul 20 21:53:27.600003 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:27.599918    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64d7f6dfcb-zmq82_e3fb0dff-5b39-431b-959b-ba6636770fe4/router/0.log"
Jul 20 21:53:28.255850 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:28.255825    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7475cd49b4-wclxq_fc7003b6-c170-420a-ba0a-d94cc9b56c36/manager/0.log"
Jul 20 21:53:28.791702 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:28.791681    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4c9892_f0fb0c79-9936-407c-a54e-ee4dc5cabc03/istio-proxy/0.log"
Jul 20 21:53:28.911679 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:28.911645    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-jp58g_70b96039-dacf-4054-8ffb-e9f20695be71/discovery/0.log"
Jul 20 21:53:29.059829 ip-10-0-128-253 kubenswrapper[2561]: I0720 21:53:29.059750    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64d7f6dfcb-zmq82_e3fb0dff-5b39-431b-959b-ba6636770fe4/router/0.log"