Jul 15 16:16:38.244910 ip-10-0-134-132 systemd[1]: Starting Kubernetes Kubelet...
Jul 15 16:16:38.611042 ip-10-0-134-132 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 16:16:38.611042 ip-10-0-134-132 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 15 16:16:38.611042 ip-10-0-134-132 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 16:16:38.611042 ip-10-0-134-132 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 15 16:16:38.611042 ip-10-0-134-132 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 16:16:38.612978 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.612886    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 15 16:16:38.617315 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617299    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 16:16:38.617315 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617314    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 16:16:38.617391 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617319    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 16:16:38.617391 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617322    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 16:16:38.617391 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617327    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 16:16:38.617391 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617330    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 16:16:38.617391 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617333    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 16:16:38.617391 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617335    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 16:16:38.617391 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617338    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 16:16:38.617391 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617341    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 16:16:38.617391 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617344    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 16:16:38.617391 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617347    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 16:16:38.617391 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617350    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 16:16:38.617391 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617360    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 16:16:38.617391 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617363    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 16:16:38.617391 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617365    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 16:16:38.617391 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617368    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 16:16:38.617391 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617371    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 16:16:38.617391 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617374    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 16:16:38.617391 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617376    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 16:16:38.617391 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617380    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 16:16:38.617391 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617383    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 16:16:38.617954 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617386    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 16:16:38.617954 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617388    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 16:16:38.617954 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617391    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 16:16:38.617954 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617394    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 16:16:38.617954 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617397    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 16:16:38.617954 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617400    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 16:16:38.617954 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617403    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 16:16:38.617954 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617406    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 16:16:38.617954 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617409    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 16:16:38.617954 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617412    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 16:16:38.617954 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617415    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 16:16:38.617954 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617417    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 16:16:38.617954 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617420    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 16:16:38.617954 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617423    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 16:16:38.617954 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617426    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 16:16:38.617954 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617428    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 16:16:38.617954 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617431    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 16:16:38.617954 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617434    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 16:16:38.617954 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617439    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 16:16:38.617954 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617444    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 16:16:38.618462 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617447    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 16:16:38.618462 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617450    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 16:16:38.618462 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617454    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 16:16:38.618462 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617457    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 16:16:38.618462 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617459    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 16:16:38.618462 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617462    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 16:16:38.618462 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617465    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 16:16:38.618462 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617467    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 16:16:38.618462 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617470    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 16:16:38.618462 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617473    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 16:16:38.618462 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617475    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 16:16:38.618462 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617478    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 16:16:38.618462 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617480    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 16:16:38.618462 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617485    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 16:16:38.618462 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617489    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 16:16:38.618462 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617492    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 16:16:38.618462 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617494    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 16:16:38.618462 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617497    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 16:16:38.618462 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617500    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 16:16:38.618951 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617503    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 16:16:38.618951 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617506    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 16:16:38.618951 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617509    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 16:16:38.618951 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617511    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 16:16:38.618951 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617514    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 16:16:38.618951 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617517    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 16:16:38.618951 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617519    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 16:16:38.618951 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617522    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 16:16:38.618951 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617525    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 16:16:38.618951 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617527    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 16:16:38.618951 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617530    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 16:16:38.618951 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617533    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 16:16:38.618951 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617535    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 16:16:38.618951 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617538    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 16:16:38.618951 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617541    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 16:16:38.618951 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617543    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 16:16:38.618951 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617546    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 16:16:38.618951 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617549    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 16:16:38.618951 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617551    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 16:16:38.619443 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617556    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 16:16:38.619443 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617560    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 16:16:38.619443 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617563    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 16:16:38.619443 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617565    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 16:16:38.619443 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617569    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 16:16:38.619443 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617572    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 16:16:38.619443 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617575    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 16:16:38.619443 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617578    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 16:16:38.619443 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617582    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 16:16:38.619443 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617584    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 16:16:38.619443 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617587    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 16:16:38.619443 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617590    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 16:16:38.619443 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617593    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 16:16:38.619443 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617596    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 16:16:38.619443 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617598    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 16:16:38.619443 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617601    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 16:16:38.619443 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617604    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 16:16:38.619443 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617607    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 16:16:38.619443 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.617610    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 16:16:38.619949 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618436    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 16:16:38.619949 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618444    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 16:16:38.619949 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618448    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 16:16:38.619949 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618455    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 16:16:38.619949 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618459    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 16:16:38.619949 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618462    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 16:16:38.619949 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618465    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 16:16:38.619949 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618471    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 16:16:38.619949 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618475    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 16:16:38.619949 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618479    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 16:16:38.619949 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618482    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 16:16:38.619949 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618485    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 16:16:38.619949 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618489    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 16:16:38.619949 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618492    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 16:16:38.619949 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618494    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 16:16:38.619949 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618497    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 16:16:38.619949 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618500    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 16:16:38.619949 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618503    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 16:16:38.619949 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618506    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 16:16:38.620491 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618509    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 16:16:38.620491 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618512    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 16:16:38.620491 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618517    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 16:16:38.620491 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618520    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 16:16:38.620491 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618523    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 16:16:38.620491 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618526    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 16:16:38.620491 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618528    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 16:16:38.620491 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618531    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 16:16:38.620491 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618534    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 16:16:38.620491 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618536    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 16:16:38.620491 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618539    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 16:16:38.620491 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618542    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 16:16:38.620491 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618545    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 16:16:38.620491 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618547    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 16:16:38.620491 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618550    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 16:16:38.620491 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618553    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 16:16:38.620491 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618555    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 16:16:38.620491 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618564    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 16:16:38.620491 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618567    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 16:16:38.620491 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618570    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 16:16:38.621025 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618573    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 16:16:38.621025 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618576    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 16:16:38.621025 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618578    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 16:16:38.621025 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618581    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 16:16:38.621025 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618584    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 16:16:38.621025 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618587    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 16:16:38.621025 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618589    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 16:16:38.621025 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618592    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 16:16:38.621025 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618595    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 16:16:38.621025 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618598    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 16:16:38.621025 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618600    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 16:16:38.621025 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618603    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 16:16:38.621025 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618606    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 16:16:38.621025 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618609    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 16:16:38.621025 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618611    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 16:16:38.621025 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618614    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 16:16:38.621025 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618616    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 16:16:38.621025 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618619    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 16:16:38.621025 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618622    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 16:16:38.621025 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618624    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 16:16:38.621557 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618627    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 16:16:38.621557 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618630    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 16:16:38.621557 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618634    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 16:16:38.621557 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618637    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 16:16:38.621557 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618640    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 16:16:38.621557 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618643    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 16:16:38.621557 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618646    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 16:16:38.621557 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618650    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 16:16:38.621557 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618653    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 16:16:38.621557 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618656    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 16:16:38.621557 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618665    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 16:16:38.621557 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618668    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 16:16:38.621557 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618670    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 16:16:38.621557 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618673    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 16:16:38.621557 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618677    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 16:16:38.621557 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618679    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 16:16:38.621557 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618682    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 16:16:38.621557 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618685    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 16:16:38.621557 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618688    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 16:16:38.622066 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618690    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 16:16:38.622066 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618694    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 16:16:38.622066 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618697    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 16:16:38.622066 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618702    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 16:16:38.622066 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618705    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 16:16:38.622066 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618708    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 16:16:38.622066 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618711    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 16:16:38.622066 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618713    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 16:16:38.622066 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618716    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 16:16:38.622066 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618719    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 16:16:38.622066 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618721    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 16:16:38.622066 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618724    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 16:16:38.622066 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618727    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 16:16:38.622066 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618729    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 16:16:38.622066 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618732    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 16:16:38.622066 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618735    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 16:16:38.622066 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618737    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 16:16:38.622066 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618740    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 16:16:38.622066 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618742    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 16:16:38.622066 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618745    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 16:16:38.622589 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.618748    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 16:16:38.622589 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619224    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 16:16:38.622589 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619230    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 16:16:38.622589 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619234    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 16:16:38.622589 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619237    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 16:16:38.622589 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619240    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 16:16:38.622589 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619243    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 16:16:38.622589 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619246    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 16:16:38.622589 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619249    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 16:16:38.622589 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619252    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 16:16:38.622589 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619255    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 16:16:38.622589 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619258    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 16:16:38.622589 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619261    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 16:16:38.622589 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619264    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 16:16:38.622589 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619267    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 16:16:38.622589 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619270    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 16:16:38.622589 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619273    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 16:16:38.622589 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619275    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 16:16:38.622589 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619278    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 16:16:38.622589 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619281    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 16:16:38.623117 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619284    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 16:16:38.623117 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619286    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 16:16:38.623117 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619289    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 16:16:38.623117 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619292    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 16:16:38.623117 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619294    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 16:16:38.623117 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619297    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 16:16:38.623117 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619301    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 16:16:38.623117 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619304    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 16:16:38.623117 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619307    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 16:16:38.623117 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619309    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 16:16:38.623117 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619312    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 16:16:38.623117 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619315    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 16:16:38.623117 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619318    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 16:16:38.623117 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619321    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 16:16:38.623117 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619323    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 16:16:38.623117 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619326    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 16:16:38.623117 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619330    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 16:16:38.623117 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619332    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 16:16:38.623117 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619335    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 16:16:38.623117 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619338    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 16:16:38.623641 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619340    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 16:16:38.623641 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619343    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 16:16:38.623641 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619346    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 16:16:38.623641 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619349    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 16:16:38.623641 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619352    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 16:16:38.623641 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619355    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 16:16:38.623641 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619357    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 16:16:38.623641 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619360    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 16:16:38.623641 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619363    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 16:16:38.623641 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619365    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 16:16:38.623641 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619368    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 16:16:38.623641 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619371    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 16:16:38.623641 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619374    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 16:16:38.623641 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619376    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 16:16:38.623641 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619381    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 16:16:38.623641 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619385    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 16:16:38.623641 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619388    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 16:16:38.623641 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619392    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 16:16:38.623641 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619394    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 16:16:38.624137 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619397    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 16:16:38.624137 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619400    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 16:16:38.624137 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619403    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 16:16:38.624137 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619405    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 16:16:38.624137 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619408    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 16:16:38.624137 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619411    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 16:16:38.624137 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619414    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 16:16:38.624137 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619417    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 16:16:38.624137 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619420    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 16:16:38.624137 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619422    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 16:16:38.624137 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619425    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 16:16:38.624137 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619428    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 16:16:38.624137 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619431    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 16:16:38.624137 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619435    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 16:16:38.624137 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619438    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 16:16:38.624137 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619440    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 16:16:38.624137 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619444    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 16:16:38.624137 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619447    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 16:16:38.624137 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619450    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 16:16:38.624137 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619453    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 16:16:38.624656 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619455    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 16:16:38.624656 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619458    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 16:16:38.624656 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619462    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 16:16:38.624656 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619466    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 16:16:38.624656 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619469    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 16:16:38.624656 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619472    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 16:16:38.624656 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619475    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 16:16:38.624656 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619478    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 16:16:38.624656 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619481    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 16:16:38.624656 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619483    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 16:16:38.624656 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619486    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 16:16:38.624656 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619489    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 16:16:38.624656 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619491    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 16:16:38.624656 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619494    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 16:16:38.624656 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619497    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 16:16:38.624656 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619500    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 16:16:38.624656 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619505    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 16:16:38.624656 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619507    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 16:16:38.624656 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619510    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 16:16:38.625171 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619513    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 16:16:38.625171 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.619515    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 16:16:38.625171 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619587    2580 flags.go:64] FLAG: --address="0.0.0.0"
Jul 15 16:16:38.625171 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619599    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 15 16:16:38.625171 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619629    2580 flags.go:64] FLAG: --anonymous-auth="true"
Jul 15 16:16:38.625171 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619637    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 15 16:16:38.625171 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619642    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 15 16:16:38.625171 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619646    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 15 16:16:38.625171 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619650    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 15 16:16:38.625171 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619654    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 15 16:16:38.625171 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619658    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 15 16:16:38.625171 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619661    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 15 16:16:38.625171 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619666    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 15 16:16:38.625171 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619669    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 15 16:16:38.625171 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619673    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 15 16:16:38.625171 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619676    2580 flags.go:64] FLAG: --cgroup-root=""
Jul 15 16:16:38.625171 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619679    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 15 16:16:38.625171 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619682    2580 flags.go:64] FLAG: --client-ca-file=""
Jul 15 16:16:38.625171 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619685    2580 flags.go:64] FLAG: --cloud-provider="external"
Jul 15 16:16:38.625171 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619688    2580 flags.go:64] FLAG: --cluster-dns="[]"
Jul 15 16:16:38.625171 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619696    2580 flags.go:64] FLAG: --cluster-domain=""
Jul 15 16:16:38.625171 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619699    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 15 16:16:38.625171 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619702    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 15 16:16:38.625171 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619706    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 15 16:16:38.625780 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619709    2580 flags.go:64] FLAG: --container-log-max-files="5"
Jul 15 16:16:38.625780 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619714    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 15 16:16:38.625780 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619717    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 15 16:16:38.625780 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619721    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 15 16:16:38.625780 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619724    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 15 16:16:38.625780 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619729    2580 flags.go:64] FLAG: --contention-profiling="false"
Jul 15 16:16:38.625780 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619739    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 15 16:16:38.625780 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619742    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 15 16:16:38.625780 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619746    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 15 16:16:38.625780 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619749    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 15 16:16:38.625780 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619753    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 15 16:16:38.625780 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619757    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 15 16:16:38.625780 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619760    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 15 16:16:38.625780 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619768    2580 flags.go:64] FLAG: --enable-load-reader="false"
Jul 15 16:16:38.625780 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619771    2580 flags.go:64] FLAG: --enable-server="true"
Jul 15 16:16:38.625780 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619774    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 15 16:16:38.625780 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619779    2580 flags.go:64] FLAG: --event-burst="100"
Jul 15 16:16:38.625780 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619783    2580 flags.go:64] FLAG: --event-qps="50"
Jul 15 16:16:38.625780 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619786    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 15 16:16:38.625780 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619789    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 15 16:16:38.625780 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619792    2580 flags.go:64] FLAG: --eviction-hard=""
Jul 15 16:16:38.625780 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619796    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 15 16:16:38.625780 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619800    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 15 16:16:38.625780 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619803    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 15 16:16:38.625780 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619806    2580 flags.go:64] FLAG: --eviction-soft=""
Jul 15 16:16:38.626468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619809    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 15 16:16:38.626468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619812    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 15 16:16:38.626468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619816    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 15 16:16:38.626468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619819    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 15 16:16:38.626468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619822    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 15 16:16:38.626468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619825    2580 flags.go:64] FLAG: --fail-swap-on="true"
Jul 15 16:16:38.626468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619840    2580 flags.go:64] FLAG: --feature-gates=""
Jul 15 16:16:38.626468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619848    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 15 16:16:38.626468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619851    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 15 16:16:38.626468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619854    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 15 16:16:38.626468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619858    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 15 16:16:38.626468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619861    2580 flags.go:64] FLAG: --healthz-port="10248"
Jul 15 16:16:38.626468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619866    2580 flags.go:64] FLAG: --help="false"
Jul 15 16:16:38.626468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619869    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-134-132.ec2.internal"
Jul 15 16:16:38.626468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619872    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 15 16:16:38.626468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619876    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 15 16:16:38.626468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619879    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 15 16:16:38.626468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619882    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 15 16:16:38.626468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619886    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 15 16:16:38.626468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619889    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 15 16:16:38.626468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619893    2580 flags.go:64] FLAG: --image-service-endpoint=""
Jul 15 16:16:38.626468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619897    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 15 16:16:38.626468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619900    2580 flags.go:64] FLAG: --kube-api-burst="100"
Jul 15 16:16:38.627100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619904    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 15 16:16:38.627100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619907    2580 flags.go:64] FLAG: --kube-api-qps="50"
Jul 15 16:16:38.627100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619910    2580 flags.go:64] FLAG: --kube-reserved=""
Jul 15 16:16:38.627100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619913    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 15 16:16:38.627100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619916    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 15 16:16:38.627100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619920    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 15 16:16:38.627100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619923    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 15 16:16:38.627100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619926    2580 flags.go:64] FLAG: --lock-file=""
Jul 15 16:16:38.627100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619928    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 15 16:16:38.627100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619932    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 15 16:16:38.627100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619935    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 15 16:16:38.627100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619940    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 15 16:16:38.627100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619944    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 15 16:16:38.627100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619947    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 15 16:16:38.627100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619950    2580 flags.go:64] FLAG: --logging-format="text"
Jul 15 16:16:38.627100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619953    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 15 16:16:38.627100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619957    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 15 16:16:38.627100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619960    2580 flags.go:64] FLAG: --manifest-url=""
Jul 15 16:16:38.627100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619963    2580 flags.go:64] FLAG: --manifest-url-header=""
Jul 15 16:16:38.627100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619967    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 15 16:16:38.627100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619970    2580 flags.go:64] FLAG: --max-open-files="1000000"
Jul 15 16:16:38.627100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619977    2580 flags.go:64] FLAG: --max-pods="110"
Jul 15 16:16:38.627100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619980    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 15 16:16:38.627100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619983    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 15 16:16:38.627100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619986    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 15 16:16:38.628221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619989    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 15 16:16:38.628221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619992    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 15 16:16:38.628221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619995    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 15 16:16:38.628221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.619998    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 15 16:16:38.628221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620007    2580 flags.go:64] FLAG: --node-status-max-images="50"
Jul 15 16:16:38.628221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620011    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 15 16:16:38.628221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620015    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 15 16:16:38.628221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620018    2580 flags.go:64] FLAG: --pod-cidr=""
Jul 15 16:16:38.628221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620021    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 15 16:16:38.628221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620027    2580 flags.go:64] FLAG: --pod-manifest-path=""
Jul 15 16:16:38.628221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620030    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 15 16:16:38.628221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620033    2580 flags.go:64] FLAG: --pods-per-core="0"
Jul 15 16:16:38.628221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620037    2580 flags.go:64] FLAG: --port="10250"
Jul 15 16:16:38.628221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620040    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 15 16:16:38.628221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620043    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00bfbb30178b2d451"
Jul 15 16:16:38.628221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620046    2580 flags.go:64] FLAG: --qos-reserved=""
Jul 15 16:16:38.628221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620049    2580 flags.go:64] FLAG: --read-only-port="10255"
Jul 15 16:16:38.628221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620052    2580 flags.go:64] FLAG: --register-node="true"
Jul 15 16:16:38.628221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620055    2580 flags.go:64] FLAG: --register-with-taints=""
Jul 15 16:16:38.628221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620059    2580 flags.go:64] FLAG: --registry-burst="10"
Jul 15 16:16:38.628221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620062    2580 flags.go:64] FLAG: --registry-qps="5"
Jul 15 16:16:38.628221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620065    2580 flags.go:64] FLAG: --reserved-cpus=""
Jul 15 16:16:38.628221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620068    2580 flags.go:64] FLAG: --reserved-memory=""
Jul 15 16:16:38.628221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620072    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 15 16:16:38.628221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620075    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 15 16:16:38.629451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620078    2580 flags.go:64] FLAG: --rotate-certificates="false"
Jul 15 16:16:38.629451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620081    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 15 16:16:38.629451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620084    2580 flags.go:64] FLAG: --runonce="false"
Jul 15 16:16:38.629451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620089    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 15 16:16:38.629451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620092    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 15 16:16:38.629451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620096    2580 flags.go:64] FLAG: --seccomp-default="false"
Jul 15 16:16:38.629451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620098    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 15 16:16:38.629451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620102    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 15 16:16:38.629451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620105    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 15 16:16:38.629451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620108    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 15 16:16:38.629451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620112    2580 flags.go:64] FLAG: --storage-driver-password="root"
Jul 15 16:16:38.629451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620115    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 15 16:16:38.629451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620119    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 15 16:16:38.629451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620123    2580 flags.go:64] FLAG: --storage-driver-user="root"
Jul 15 16:16:38.629451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620126    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 15 16:16:38.629451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620129    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 15 16:16:38.629451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620132    2580 flags.go:64] FLAG: --system-cgroups=""
Jul 15 16:16:38.629451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620135    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 15 16:16:38.629451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620141    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 15 16:16:38.629451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620144    2580 flags.go:64] FLAG: --tls-cert-file=""
Jul 15 16:16:38.629451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620147    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 15 16:16:38.629451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620151    2580 flags.go:64] FLAG: --tls-min-version=""
Jul 15 16:16:38.629451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620154    2580 flags.go:64] FLAG: --tls-private-key-file=""
Jul 15 16:16:38.629451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620157    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 15 16:16:38.629451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620160    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 15 16:16:38.630197 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620163    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 15 16:16:38.630197 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620166    2580 flags.go:64] FLAG: --v="2"
Jul 15 16:16:38.630197 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620170    2580 flags.go:64] FLAG: --version="false"
Jul 15 16:16:38.630197 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620174    2580 flags.go:64] FLAG: --vmodule=""
Jul 15 16:16:38.630197 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620178    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 15 16:16:38.630197 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.620181    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 15 16:16:38.630197 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620318    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 16:16:38.630197 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620323    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 16:16:38.630197 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620326    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 16:16:38.630197 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620329    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 16:16:38.630197 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620334    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 16:16:38.630197 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620337    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 16:16:38.630197 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620340    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 16:16:38.630197 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620342    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 16:16:38.630197 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620346    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 16:16:38.630197 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620349    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 16:16:38.630197 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620351    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 16:16:38.630197 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620354    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 16:16:38.630197 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620357    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 16:16:38.630197 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620361    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 16:16:38.630197 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620364    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 16:16:38.631067 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620367    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 16:16:38.631067 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620377    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 16:16:38.631067 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620380    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 16:16:38.631067 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620382    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 16:16:38.631067 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620385    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 16:16:38.631067 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620388    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 16:16:38.631067 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620390    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 16:16:38.631067 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620393    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 16:16:38.631067 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620396    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 16:16:38.631067 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620398    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 16:16:38.631067 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620401    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 16:16:38.631067 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620404    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 16:16:38.631067 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620406    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 16:16:38.631067 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620409    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 16:16:38.631067 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620412    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 16:16:38.631067 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620415    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 16:16:38.631067 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620418    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 16:16:38.631067 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620420    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 16:16:38.631067 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620423    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 16:16:38.631067 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620426    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 16:16:38.632011 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620428    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 16:16:38.632011 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620433    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 16:16:38.632011 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620436    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 16:16:38.632011 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620438    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 16:16:38.632011 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620441    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 16:16:38.632011 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620444    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 16:16:38.632011 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620446    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 16:16:38.632011 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620449    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 16:16:38.632011 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620452    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 16:16:38.632011 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620455    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 16:16:38.632011 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620460    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 16:16:38.632011 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620463    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 16:16:38.632011 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620466    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 16:16:38.632011 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620468    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 16:16:38.632011 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620473    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 16:16:38.632011 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620475    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 16:16:38.632011 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620478    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 16:16:38.632011 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620481    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 16:16:38.632011 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620484    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 16:16:38.632011 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620487    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 16:16:38.632819 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620490    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 16:16:38.632819 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620495    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 16:16:38.632819 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620499    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 16:16:38.632819 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620502    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 16:16:38.632819 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620506    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 16:16:38.632819 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620509    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 16:16:38.632819 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620512    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 16:16:38.632819 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620515    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 16:16:38.632819 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620517    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 16:16:38.632819 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620520    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 16:16:38.632819 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620523    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 16:16:38.632819 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620526    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 16:16:38.632819 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620528    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 16:16:38.632819 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620532    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 16:16:38.632819 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620535    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 16:16:38.632819 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620538    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 16:16:38.632819 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620541    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 16:16:38.632819 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620544    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 16:16:38.632819 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620546    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 16:16:38.633374 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620549    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 16:16:38.633374 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620552    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 16:16:38.633374 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620555    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 16:16:38.633374 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620559    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 16:16:38.633374 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620561    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 16:16:38.633374 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620564    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 16:16:38.633374 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620567    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 16:16:38.633374 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620570    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 16:16:38.633374 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620573    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 16:16:38.633374 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620575    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 16:16:38.633374 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620578    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 16:16:38.633374 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620581    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 16:16:38.633374 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620585    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 16:16:38.633374 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620589    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 16:16:38.633374 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620592    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 16:16:38.633374 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620595    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 16:16:38.633374 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620600    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 16:16:38.633374 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620603    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 16:16:38.633374 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620606    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 16:16:38.633979 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620609    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 16:16:38.633979 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620613    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 16:16:38.633979 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620616    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 16:16:38.633979 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620618    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 16:16:38.633979 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620621    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 16:16:38.633979 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.620624    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 16:16:38.633979 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.621497    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 16:16:38.633979 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.629559    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 15 16:16:38.633979 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.629580    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 15 16:16:38.633979 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.630400    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 15 16:16:38.633979 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.630419    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 15 16:16:38.633979 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630513    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 16:16:38.633979 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630523    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 16:16:38.633979 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630529    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 16:16:38.634350 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630533    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 16:16:38.634350 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630538    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 16:16:38.634350 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630543    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 16:16:38.634350 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630547    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 16:16:38.634350 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630553    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 16:16:38.634350 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630559    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 16:16:38.634350 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630563    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 16:16:38.634350 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630568    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 16:16:38.634350 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630572    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 16:16:38.634350 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630577    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 16:16:38.634350 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630582    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 16:16:38.634350 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630586    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 16:16:38.634350 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630591    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 16:16:38.634350 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630595    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 16:16:38.634350 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630600    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 16:16:38.634350 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630604    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 16:16:38.634350 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630609    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 16:16:38.634350 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630614    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 16:16:38.634350 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630618    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 16:16:38.634845 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630622    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 16:16:38.634845 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630628    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 16:16:38.634845 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630632    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 16:16:38.634845 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630637    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 16:16:38.634845 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630641    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 16:16:38.634845 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630646    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 16:16:38.634845 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630651    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 16:16:38.634845 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630657    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 16:16:38.634845 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630662    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 16:16:38.634845 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630667    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 16:16:38.634845 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630672    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 16:16:38.634845 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630676    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 16:16:38.634845 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630680    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 16:16:38.634845 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630685    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 16:16:38.634845 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630689    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 16:16:38.634845 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630694    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 16:16:38.634845 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630699    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 16:16:38.634845 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630703    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 16:16:38.634845 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630708    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 16:16:38.634845 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630712    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 16:16:38.635387 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630716    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 16:16:38.635387 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630721    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 16:16:38.635387 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630725    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 16:16:38.635387 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630730    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 16:16:38.635387 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630735    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 16:16:38.635387 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630740    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 16:16:38.635387 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630745    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 16:16:38.635387 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630749    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 16:16:38.635387 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630754    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 16:16:38.635387 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630758    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 16:16:38.635387 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630763    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 16:16:38.635387 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630767    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 16:16:38.635387 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630772    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 16:16:38.635387 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630776    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 16:16:38.635387 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630780    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 16:16:38.635387 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630801    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 16:16:38.635387 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630806    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 16:16:38.635387 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630811    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 16:16:38.635387 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630816    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 16:16:38.635387 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630822    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 16:16:38.635936 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630827    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 16:16:38.635936 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630832    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 16:16:38.635936 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630839    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 16:16:38.635936 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630846    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 16:16:38.635936 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630851    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 16:16:38.635936 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630855    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 16:16:38.635936 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630861    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 16:16:38.635936 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630866    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 16:16:38.635936 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630870    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 16:16:38.635936 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630875    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 16:16:38.635936 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630879    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 16:16:38.635936 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630883    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 16:16:38.635936 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630887    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 16:16:38.635936 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630892    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 16:16:38.635936 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630896    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 16:16:38.635936 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630900    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 16:16:38.635936 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630905    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 16:16:38.635936 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630910    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 16:16:38.635936 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630915    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 16:16:38.635936 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630919    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 16:16:38.636459 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630923    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 16:16:38.636459 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630928    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 16:16:38.636459 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630932    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 16:16:38.636459 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630936    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 16:16:38.636459 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630943    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 16:16:38.636459 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630949    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 16:16:38.636459 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630954    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 16:16:38.636459 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630959    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 16:16:38.636459 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630965    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 16:16:38.636459 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630971    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 16:16:38.636459 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630976    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 16:16:38.636459 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630982    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 16:16:38.636459 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630988    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 16:16:38.636459 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630994    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 16:16:38.636459 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.630999    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 16:16:38.636459 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631004    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 16:16:38.636459 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631009    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 16:16:38.637005 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.631018    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 16:16:38.637005 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631219    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 16:16:38.637005 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631230    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 16:16:38.637005 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631237    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 16:16:38.637005 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631242    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 16:16:38.637005 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631246    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 16:16:38.637005 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631252    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 16:16:38.637005 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631258    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 16:16:38.637005 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631262    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 16:16:38.637005 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631267    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 16:16:38.637005 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631271    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 16:16:38.637005 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631276    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 16:16:38.637005 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631281    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 16:16:38.637005 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631285    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 16:16:38.637005 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631289    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 16:16:38.637400 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631294    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 16:16:38.637400 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631298    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 16:16:38.637400 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631303    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 16:16:38.637400 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631308    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 16:16:38.637400 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631312    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 16:16:38.637400 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631316    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 16:16:38.637400 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631321    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 16:16:38.637400 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631325    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 16:16:38.637400 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631330    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 16:16:38.637400 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631334    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 16:16:38.637400 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631339    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 16:16:38.637400 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631344    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 16:16:38.637400 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631349    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 16:16:38.637400 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631353    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 16:16:38.637400 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631408    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 16:16:38.637400 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631470    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 16:16:38.637400 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631478    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 16:16:38.637400 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631484    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 16:16:38.637400 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631489    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 16:16:38.637965 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631494    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 16:16:38.637965 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631500    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 16:16:38.637965 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631504    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 16:16:38.637965 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631508    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 16:16:38.637965 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631513    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 16:16:38.637965 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631518    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 16:16:38.637965 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631524    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 16:16:38.637965 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631529    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 16:16:38.637965 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.631834    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 16:16:38.637965 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632308    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 16:16:38.637965 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632318    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 16:16:38.637965 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632325    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 16:16:38.637965 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632330    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 16:16:38.637965 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632336    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 16:16:38.637965 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632342    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 16:16:38.637965 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632348    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 16:16:38.637965 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632353    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 16:16:38.637965 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632358    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 16:16:38.637965 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632363    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 16:16:38.638497 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632368    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 16:16:38.638497 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632379    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 16:16:38.638497 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632384    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 16:16:38.638497 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632389    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 16:16:38.638497 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632394    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 16:16:38.638497 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632399    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 16:16:38.638497 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632404    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 16:16:38.638497 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632410    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 16:16:38.638497 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632414    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 16:16:38.638497 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632419    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 16:16:38.638497 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632424    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 16:16:38.638497 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632429    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 16:16:38.638497 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632434    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 16:16:38.638497 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632438    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 16:16:38.638497 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632447    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 16:16:38.638497 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632452    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 16:16:38.638497 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632458    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 16:16:38.638497 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632462    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 16:16:38.638497 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632468    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 16:16:38.638497 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632472    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 16:16:38.639031 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632477    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 16:16:38.639031 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632482    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 16:16:38.639031 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632487    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 16:16:38.639031 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632494    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 16:16:38.639031 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632499    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 16:16:38.639031 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632503    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 16:16:38.639031 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632513    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 16:16:38.639031 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632519    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 16:16:38.639031 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632524    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 16:16:38.639031 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632528    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 16:16:38.639031 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632533    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 16:16:38.639031 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632538    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 16:16:38.639031 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632543    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 16:16:38.639031 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632547    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 16:16:38.639031 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632552    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 16:16:38.639031 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632556    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 16:16:38.639031 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632561    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 16:16:38.639031 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632565    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 16:16:38.639031 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632570    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 16:16:38.639031 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632580    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 16:16:38.639551 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632584    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 16:16:38.639551 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632589    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 16:16:38.639551 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632594    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 16:16:38.639551 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632599    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 16:16:38.639551 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632603    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 16:16:38.639551 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632608    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 16:16:38.639551 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:38.632613    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 16:16:38.639551 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.632622    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 16:16:38.639551 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.633171    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 15 16:16:38.639551 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.635412    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 15 16:16:38.639551 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.636678    2580 server.go:1019] "Starting client certificate rotation"
Jul 15 16:16:38.639551 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.636786    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 16:16:38.639551 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.637519    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 16:16:38.639924 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.639184    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 16:16:38.642084 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.642066    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 16:16:38.650736 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.650718    2580 log.go:25] "Validated CRI v1 runtime API"
Jul 15 16:16:38.654113 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.654097    2580 log.go:25] "Validated CRI v1 image API"
Jul 15 16:16:38.655038 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.655018    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 15 16:16:38.657412 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.657392    2580 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 d2399311-60bb-41ae-ae66-4f17bbfd1b03:/dev/nvme0n1p4 e1d55794-fccc-479a-bf15-6a51e0361ece:/dev/nvme0n1p3]
Jul 15 16:16:38.657476 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.657411    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 15 16:16:38.662033 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.661917    2580 manager.go:216] Machine: {Timestamp:2026-07-15 16:16:38.661402999 +0000 UTC m=+0.308443529 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099801 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20b97965e4d0149c0d4dcc99f6abf7 SystemUUID:ec20b979-65e4-d014-9c0d-4dcc99f6abf7 BootID:b549bb2a-06b4-435e-89bf-0c8aafa10047 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:41:59:3a:fc:23 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:41:59:3a:fc:23 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:62:a2:d1:a4:95 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 15 16:16:38.662033 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.662032    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 15 16:16:38.662166 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.662154    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 15 16:16:38.663264 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.663242    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 15 16:16:38.663415 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.663267    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-132.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 15 16:16:38.664042 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.664030    2580 topology_manager.go:138] "Creating topology manager with none policy"
Jul 15 16:16:38.664076 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.664045    2580 container_manager_linux.go:309] "Creating device plugin manager"
Jul 15 16:16:38.664076 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.664058    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 16:16:38.665041 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.665029    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 16:16:38.666383 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.666372    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 15 16:16:38.669328 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.669316    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 16:16:38.669465 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.669455    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 15 16:16:38.670095 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.670084    2580 kubelet.go:486] "Attempting to sync node with API server"
Jul 15 16:16:38.670132 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.670101    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 15 16:16:38.670132 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.670126    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 15 16:16:38.670187 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.670141    2580 kubelet.go:398] "Adding apiserver pod source"
Jul 15 16:16:38.670187 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.670155    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 15 16:16:38.670640 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.670624    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 16:16:38.670705 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.670651    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 16:16:38.674411 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.674299    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 15 16:16:38.676228 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.676209    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 15 16:16:38.676312 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.676251    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 15 16:16:38.676498 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.676479    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 15 16:16:38.676595 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.676504    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 15 16:16:38.676595 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.676515    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 15 16:16:38.676595 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.676524    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 15 16:16:38.676595 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.676538    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 15 16:16:38.676595 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.676547    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 15 16:16:38.676939 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.676928    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 15 16:16:38.676939 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.676938    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 15 16:16:38.677009 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.676945    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 15 16:16:38.677009 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.676952    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 15 16:16:38.677009 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.676961    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 15 16:16:38.677009 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.676969    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 15 16:16:38.678263 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.678249    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 15 16:16:38.678299 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.678268    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 15 16:16:38.681683 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.681668    2580 server.go:1268] "Started kubelet"
Jul 15 16:16:38.681759 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.681722    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 15 16:16:38.681917 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.681861    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 15 16:16:38.681965 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.681941    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 15 16:16:38.682947 ip-10-0-134-132 systemd[1]: Started Kubernetes Kubelet.
Jul 15 16:16:38.683255 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.683237    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 15 16:16:38.684460 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.684446    2580 server.go:310] "Adding debug handlers to kubelet server"
Jul 15 16:16:38.687053 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:38.687032    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 15 16:16:38.688471 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.688449    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-132.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 15 16:16:38.688800 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.688772    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 15 16:16:38.688886 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.688783    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 15 16:16:38.688886 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:38.688882    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-132.ec2.internal\" not found"
Jul 15 16:16:38.689135 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:38.689099    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 15 16:16:38.689268 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:38.689235    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-132.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 15 16:16:38.689442 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.689427    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 15 16:16:38.689507 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.689447    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 15 16:16:38.689561 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:38.688491    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-132.ec2.internal.18c282c82ec16dc0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-132.ec2.internal,UID:ip-10-0-134-132.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-132.ec2.internal,},FirstTimestamp:2026-07-15 16:16:38.681636288 +0000 UTC m=+0.328676817,LastTimestamp:2026-07-15 16:16:38.681636288 +0000 UTC m=+0.328676817,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-132.ec2.internal,}"
Jul 15 16:16:38.689561 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.689536    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 15 16:16:38.689695 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.689579    2580 reconstruct.go:97] "Volume reconstruction finished"
Jul 15 16:16:38.689695 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.689591    2580 reconciler.go:29] "Reconciler: start to sync state"
Jul 15 16:16:38.690084 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.690069    2580 factory.go:153] Registering CRI-O factory
Jul 15 16:16:38.690175 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.690088    2580 factory.go:223] Registration of the crio container factory successfully
Jul 15 16:16:38.690175 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.690155    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 15 16:16:38.690175 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.690168    2580 factory.go:55] Registering systemd factory
Jul 15 16:16:38.690175 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.690177    2580 factory.go:223] Registration of the systemd container factory successfully
Jul 15 16:16:38.690369 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.690199    2580 factory.go:103] Registering Raw factory
Jul 15 16:16:38.690369 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.690213    2580 manager.go:1196] Started watching for new ooms in manager
Jul 15 16:16:38.690598 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.690584    2580 manager.go:349] Starting recovery of all containers
Jul 15 16:16:38.702136 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.702111    2580 manager.go:354] Recovery completed
Jul 15 16:16:38.707760 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.707740    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 16:16:38.707760 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:38.707740    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 15 16:16:38.707926 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:38.707819    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-132.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 15 16:16:38.709649 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.709631    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-132.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 16:16:38.709714 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.709661    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-132.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 16:16:38.709714 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.709672    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-132.ec2.internal" event="NodeHasSufficientPID"
Jul 15 16:16:38.710312 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.710295    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 15 16:16:38.710312 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.710309    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 15 16:16:38.710442 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.710325    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 16:16:38.710442 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.710311    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 16:16:38.711126 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.711112    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-132.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 16:16:38.711183 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.711139    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-132.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 16:16:38.711183 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.711152    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-132.ec2.internal" event="NodeHasSufficientPID"
Jul 15 16:16:38.711645 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.711628    2580 policy_none.go:49] "None policy: Start"
Jul 15 16:16:38.711645 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.711647    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 15 16:16:38.711776 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.711663    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 15 16:16:38.712801 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.712789    2580 policy_none.go:47] "Start"
Jul 15 16:16:38.722489 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.722468    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 16:16:38.725183 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:38.725103    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-132.ec2.internal.18c282c8306cd9cd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-132.ec2.internal,UID:ip-10-0-134-132.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-132.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-132.ec2.internal,},FirstTimestamp:2026-07-15 16:16:38.709647821 +0000 UTC m=+0.356688351,LastTimestamp:2026-07-15 16:16:38.709647821 +0000 UTC m=+0.356688351,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-132.ec2.internal,}"
Jul 15 16:16:38.730017 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:38.729948    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-132.ec2.internal.18c282c8306d2495  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-132.ec2.internal,UID:ip-10-0-134-132.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-132.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-132.ec2.internal,},FirstTimestamp:2026-07-15 16:16:38.709666965 +0000 UTC m=+0.356707495,LastTimestamp:2026-07-15 16:16:38.709666965 +0000 UTC m=+0.356707495,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-132.ec2.internal,}"
Jul 15 16:16:38.743678 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:38.743607    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-132.ec2.internal.18c282c8306d4a33  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-132.ec2.internal,UID:ip-10-0-134-132.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-134-132.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-134-132.ec2.internal,},FirstTimestamp:2026-07-15 16:16:38.709676595 +0000 UTC m=+0.356717124,LastTimestamp:2026-07-15 16:16:38.709676595 +0000 UTC m=+0.356717124,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-132.ec2.internal,}"
Jul 15 16:16:38.747448 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.747434    2580 manager.go:337] "Starting Device Plugin manager"
Jul 15 16:16:38.747544 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:38.747468    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 15 16:16:38.747544 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.747478    2580 server.go:87] "Starting device plugin registration server"
Jul 15 16:16:38.747916 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.747762    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 15 16:16:38.747967 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.747923    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 15 16:16:38.748073 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.748051    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 15 16:16:38.748166 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.748152    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 15 16:16:38.748213 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.748166    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 15 16:16:38.748718 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:38.748698    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 15 16:16:38.748802 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:38.748757    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-132.ec2.internal\" not found"
Jul 15 16:16:38.762475 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:38.762396    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-132.ec2.internal.18c282c8306cd9cd\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-132.ec2.internal.18c282c8306cd9cd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-132.ec2.internal,UID:ip-10-0-134-132.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-132.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-132.ec2.internal,},FirstTimestamp:2026-07-15 16:16:38.709647821 +0000 UTC m=+0.356688351,LastTimestamp:2026-07-15 16:16:38.711125867 +0000 UTC m=+0.358166397,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-132.ec2.internal,}"
Jul 15 16:16:38.788284 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:38.788209    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-132.ec2.internal.18c282c8306d2495\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-132.ec2.internal.18c282c8306d2495  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-132.ec2.internal,UID:ip-10-0-134-132.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-132.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-132.ec2.internal,},FirstTimestamp:2026-07-15 16:16:38.709666965 +0000 UTC m=+0.356707495,LastTimestamp:2026-07-15 16:16:38.711146443 +0000 UTC m=+0.358187115,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-132.ec2.internal,}"
Jul 15 16:16:38.800000 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.799977    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gnjnj"
Jul 15 16:16:38.807781 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:38.807659    2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-132.ec2.internal.18c282c8306d4a33\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-132.ec2.internal.18c282c8306d4a33  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-132.ec2.internal,UID:ip-10-0-134-132.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-134-132.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-134-132.ec2.internal,},FirstTimestamp:2026-07-15 16:16:38.709676595 +0000 UTC m=+0.356717124,LastTimestamp:2026-07-15 16:16:38.711156704 +0000 UTC m=+0.358197234,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-132.ec2.internal,}"
Jul 15 16:16:38.816425 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.816408    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gnjnj"
Jul 15 16:16:38.848701 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.848670    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 16:16:38.849604 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.849582    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-132.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 16:16:38.849698 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.849625    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-132.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 16:16:38.849698 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.849638    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-132.ec2.internal" event="NodeHasSufficientPID"
Jul 15 16:16:38.849698 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.849666    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-132.ec2.internal"
Jul 15 16:16:38.871341 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.871307    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 15 16:16:38.872633 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.872616    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 15 16:16:38.872723 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.872661    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 15 16:16:38.872723 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.872689    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 15 16:16:38.872818 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:38.872732    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 15 16:16:38.875978 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.875958    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 16:16:38.877604 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.877590    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-132.ec2.internal"
Jul 15 16:16:38.877656 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:38.877612    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-132.ec2.internal\": node \"ip-10-0-134-132.ec2.internal\" not found"
Jul 15 16:16:38.973794 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.973710    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-132.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-132.ec2.internal"]
Jul 15 16:16:38.973982 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.973861    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 16:16:38.975139 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.975120    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-132.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 16:16:38.975259 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.975156    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-132.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 16:16:38.975259 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.975171    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-132.ec2.internal" event="NodeHasSufficientPID"
Jul 15 16:16:38.976671 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.976655    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 16:16:38.976804 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.976789    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-132.ec2.internal"
Jul 15 16:16:38.976870 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.976819    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 16:16:38.977443 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.977427    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-132.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 16:16:38.977518 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.977431    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-132.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 16:16:38.977518 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.977477    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-132.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 16:16:38.977518 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.977488    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-132.ec2.internal" event="NodeHasSufficientPID"
Jul 15 16:16:38.977634 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.977457    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-132.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 16:16:38.977634 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.977552    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-132.ec2.internal" event="NodeHasSufficientPID"
Jul 15 16:16:38.978902 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.978887    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-132.ec2.internal"
Jul 15 16:16:38.978970 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.978921    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 16:16:38.979671 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.979654    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-132.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 16:16:38.979775 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.979678    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-132.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 16:16:38.979775 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.979693    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-132.ec2.internal" event="NodeHasSufficientPID"
Jul 15 16:16:38.991588 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.991568    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4c8d0721ab10fafc7cda1fc3cfef7c2b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-132.ec2.internal\" (UID: \"4c8d0721ab10fafc7cda1fc3cfef7c2b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-132.ec2.internal"
Jul 15 16:16:38.991676 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.991595    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c8d0721ab10fafc7cda1fc3cfef7c2b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-132.ec2.internal\" (UID: \"4c8d0721ab10fafc7cda1fc3cfef7c2b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-132.ec2.internal"
Jul 15 16:16:38.991676 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:38.991616    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dc2084eaf308dac38296e58e829ee36f-config\") pod \"kube-apiserver-proxy-ip-10-0-134-132.ec2.internal\" (UID: \"dc2084eaf308dac38296e58e829ee36f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-132.ec2.internal"
Jul 15 16:16:38.995402 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:38.995383    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-132.ec2.internal\" not found" node="ip-10-0-134-132.ec2.internal"
Jul 15 16:16:38.999383 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:38.999366    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-132.ec2.internal\" not found" node="ip-10-0-134-132.ec2.internal"
Jul 15 16:16:39.033392 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:39.033370    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-132.ec2.internal\" not found"
Jul 15 16:16:39.091816 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:39.091704    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4c8d0721ab10fafc7cda1fc3cfef7c2b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-132.ec2.internal\" (UID: \"4c8d0721ab10fafc7cda1fc3cfef7c2b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-132.ec2.internal"
Jul 15 16:16:39.091816 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:39.091753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c8d0721ab10fafc7cda1fc3cfef7c2b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-132.ec2.internal\" (UID: \"4c8d0721ab10fafc7cda1fc3cfef7c2b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-132.ec2.internal"
Jul 15 16:16:39.091816 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:39.091789    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dc2084eaf308dac38296e58e829ee36f-config\") pod \"kube-apiserver-proxy-ip-10-0-134-132.ec2.internal\" (UID: \"dc2084eaf308dac38296e58e829ee36f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-132.ec2.internal"
Jul 15 16:16:39.091816 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:39.091821    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dc2084eaf308dac38296e58e829ee36f-config\") pod \"kube-apiserver-proxy-ip-10-0-134-132.ec2.internal\" (UID: \"dc2084eaf308dac38296e58e829ee36f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-132.ec2.internal"
Jul 15 16:16:39.092036 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:39.091825    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c8d0721ab10fafc7cda1fc3cfef7c2b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-132.ec2.internal\" (UID: \"4c8d0721ab10fafc7cda1fc3cfef7c2b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-132.ec2.internal"
Jul 15 16:16:39.092036 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:39.091826    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4c8d0721ab10fafc7cda1fc3cfef7c2b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-132.ec2.internal\" (UID: \"4c8d0721ab10fafc7cda1fc3cfef7c2b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-132.ec2.internal"
Jul 15 16:16:39.135216 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:39.135178    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-132.ec2.internal\" not found"
Jul 15 16:16:39.235611 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:39.235569    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-132.ec2.internal\" not found"
Jul 15 16:16:39.297034 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:39.296982    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-132.ec2.internal"
Jul 15 16:16:39.302606 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:39.302588    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-132.ec2.internal"
Jul 15 16:16:39.336314 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:39.336284    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-132.ec2.internal\" not found"
Jul 15 16:16:39.439718 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:39.439612    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-132.ec2.internal\" not found"
Jul 15 16:16:39.540529 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:39.540475    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-132.ec2.internal\" not found"
Jul 15 16:16:39.636008 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:39.635970    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 15 16:16:39.636489 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:39.636147    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 16:16:39.641565 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:39.641547    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-132.ec2.internal\" not found"
Jul 15 16:16:39.702362 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:39.702293    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 16:16:39.741889 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:39.741854    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-132.ec2.internal\" not found"
Jul 15 16:16:39.752958 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:39.752936    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 16:16:39.754007 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:39.753991    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-132.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 16:16:39.754101 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:39.754023    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-132.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 16:16:39.754101 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:39.754039    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-132.ec2.internal" event="NodeHasSufficientPID"
Jul 15 16:16:39.819517 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:39.819474    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 16:11:38 +0000 UTC" deadline="2028-01-29 13:23:52.125368714 +0000 UTC"
Jul 15 16:16:39.819517 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:39.819512    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13509h7m12.305860498s"
Jul 15 16:16:39.842732 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:39.842701    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-132.ec2.internal\" not found"
Jul 15 16:16:39.854251 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:39.854219    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 16:16:39.886466 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:39.886437    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 16:16:39.889045 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:39.889021    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-132.ec2.internal"
Jul 15 16:16:39.914278 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:39.914251    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 16:16:39.915277 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:39.915262    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-132.ec2.internal"
Jul 15 16:16:39.937036 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:39.937013    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 16:16:39.938044 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:39.938018    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc2084eaf308dac38296e58e829ee36f.slice/crio-5a0b0798f574a6d95887a44d1c92a46e8744f19266dae4065830f7e3c65050f4 WatchSource:0}: Error finding container 5a0b0798f574a6d95887a44d1c92a46e8744f19266dae4065830f7e3c65050f4: Status 404 returned error can't find the container with id 5a0b0798f574a6d95887a44d1c92a46e8744f19266dae4065830f7e3c65050f4
Jul 15 16:16:39.943060 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:39.943033    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 16:16:39.969838 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:39.969811    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c8d0721ab10fafc7cda1fc3cfef7c2b.slice/crio-df643003e2e7d2ad63e64443937996b74ea8c37afdb5fac9da9fc586df102728 WatchSource:0}: Error finding container df643003e2e7d2ad63e64443937996b74ea8c37afdb5fac9da9fc586df102728: Status 404 returned error can't find the container with id df643003e2e7d2ad63e64443937996b74ea8c37afdb5fac9da9fc586df102728
Jul 15 16:16:40.672655 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.672627    2580 apiserver.go:52] "Watching apiserver"
Jul 15 16:16:40.689844 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.689814    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 15 16:16:40.700016 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.699986    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 15 16:16:40.701102 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.701064    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-vzmbb","openshift-dns/node-resolver-hk4xk","openshift-image-registry/node-ca-7tktr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-132.ec2.internal","openshift-multus/multus-4srwh","openshift-multus/multus-additional-cni-plugins-6zrmr","openshift-multus/network-metrics-daemon-g9k24","openshift-network-diagnostics/network-check-target-t6fc5","openshift-network-operator/iptables-alerter-dt445","openshift-ovn-kubernetes/ovnkube-node-295d8","kube-system/global-pull-secret-syncer-c75b9","kube-system/konnectivity-agent-gzwh7","kube-system/kube-apiserver-proxy-ip-10-0-134-132.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b"]
Jul 15 16:16:40.702653 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.702627    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.703969 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.703949    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hk4xk"
Jul 15 16:16:40.705745 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.705720    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7tktr"
Jul 15 16:16:40.706015 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.705996    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 15 16:16:40.706279 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.706259    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-c5qhj\""
Jul 15 16:16:40.706380 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.706337    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 15 16:16:40.707037 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.706841    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.708570 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.708541    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:40.708691 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.708673    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5wphh\""
Jul 15 16:16:40.708972 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.708957    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 15 16:16:40.709114 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.709094    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 15 16:16:40.709861 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.709842    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:16:40.709962 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:40.709916    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9k24" podUID="a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd"
Jul 15 16:16:40.711072 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.711052    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:16:40.711161 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:40.711123    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t6fc5" podUID="422a9314-d5df-4554-ae6d-2a21856dcb6c"
Jul 15 16:16:40.712801 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.712787    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dt445"
Jul 15 16:16:40.714045 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.714027    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.715594 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.715576    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:16:40.715681 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:40.715654    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75b9" podUID="7549f9c6-5988-4771-ae95-99e674252273"
Jul 15 16:16:40.716968 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.716944    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gzwh7"
Jul 15 16:16:40.718431 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.718387    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b"
Jul 15 16:16:40.722232 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.722210    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 15 16:16:40.722323 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.722270    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 15 16:16:40.722379 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.722322    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 15 16:16:40.722429 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.722274    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 15 16:16:40.722482 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.722210    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 15 16:16:40.722577 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.722561    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-r2758\""
Jul 15 16:16:40.722744 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.722640    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bl22k\""
Jul 15 16:16:40.722873 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.722831    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 15 16:16:40.723087 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.722939    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 15 16:16:40.723087 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.722831    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 15 16:16:40.723087 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.723019    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 15 16:16:40.723274 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.723180    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 15 16:16:40.723274 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.723261    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ls8w2\""
Jul 15 16:16:40.723442 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.723401    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 15 16:16:40.723532 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.723444    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 15 16:16:40.723532 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.723488    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 15 16:16:40.723640 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.723571    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 15 16:16:40.723640 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.723609    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 15 16:16:40.727395 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.724243    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-84c9g\""
Jul 15 16:16:40.727395 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.724719    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hcm2g\""
Jul 15 16:16:40.727395 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.725273    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nngdk\""
Jul 15 16:16:40.727395 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.726194    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 15 16:16:40.727395 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.726336    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 15 16:16:40.727395 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.726487    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 15 16:16:40.736909 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.736887    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 16:16:40.737268 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.737250    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 15 16:16:40.740365 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.740324    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 15 16:16:40.740365 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.740336    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 15 16:16:40.742934 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.742915    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-nbbnk\""
Jul 15 16:16:40.743282 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.743262    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 15 16:16:40.743397 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.743269    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 15 16:16:40.790947 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.790922    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 15 16:16:40.803066 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803043    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-etc-kubernetes\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.803173 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803076    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-var-lib-kubelet\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.803173 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803099    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6019a7f5-5848-4017-a06d-c728a14c4f31-host\") pod \"node-ca-7tktr\" (UID: \"6019a7f5-5848-4017-a06d-c728a14c4f31\") " pod="openshift-image-registry/node-ca-7tktr"
Jul 15 16:16:40.803173 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803115    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d7494c97-f5f3-47af-a6c7-7f939c53bc39-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6zrmr\" (UID: \"d7494c97-f5f3-47af-a6c7-7f939c53bc39\") " pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:40.803173 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803165    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-system-cni-dir\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.803374 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803207    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-cnibin\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.803374 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803255    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-host-kubelet\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.803374 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803277    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-etc-openvswitch\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.803374 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803292    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/52c1f0a0-9858-45d7-bd19-7d705e9c4e42-hosts-file\") pod \"node-resolver-hk4xk\" (UID: \"52c1f0a0-9858-45d7-bd19-7d705e9c4e42\") " pod="openshift-dns/node-resolver-hk4xk"
Jul 15 16:16:40.803374 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803308    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d7494c97-f5f3-47af-a6c7-7f939c53bc39-system-cni-dir\") pod \"multus-additional-cni-plugins-6zrmr\" (UID: \"d7494c97-f5f3-47af-a6c7-7f939c53bc39\") " pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:40.803374 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803333    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-hostroot\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.803374 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803364    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-host-run-netns\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.803677 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803405    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf4vk\" (UniqueName: \"kubernetes.io/projected/46356463-bfb1-42b8-8871-71357489a95b-kube-api-access-tf4vk\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.803677 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803427    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b4b9d03d-1841-4392-84d9-e97c648a1eec-etc-selinux\") pod \"aws-ebs-csi-driver-node-tbb4b\" (UID: \"b4b9d03d-1841-4392-84d9-e97c648a1eec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b"
Jul 15 16:16:40.803677 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803444    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-host-var-lib-cni-multus\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.803677 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803480    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-host-run-multus-certs\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.803677 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803503    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/46356463-bfb1-42b8-8871-71357489a95b-ovnkube-config\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.803677 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803530    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9106d273-27df-4be5-84d0-c22aff403246-tmp\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.803677 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803564    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7549f9c6-5988-4771-ae95-99e674252273-kubelet-config\") pod \"global-pull-secret-syncer-c75b9\" (UID: \"7549f9c6-5988-4771-ae95-99e674252273\") " pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:16:40.803677 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803600    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-host\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.803677 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803626    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d7494c97-f5f3-47af-a6c7-7f939c53bc39-cni-binary-copy\") pod \"multus-additional-cni-plugins-6zrmr\" (UID: \"d7494c97-f5f3-47af-a6c7-7f939c53bc39\") " pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:40.803677 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803656    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-run-systemd\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.804129 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803691    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-etc-sysconfig\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.804129 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803724    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9106d273-27df-4be5-84d0-c22aff403246-etc-tuned\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.804129 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803751    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x66vw\" (UniqueName: \"kubernetes.io/projected/52c1f0a0-9858-45d7-bd19-7d705e9c4e42-kube-api-access-x66vw\") pod \"node-resolver-hk4xk\" (UID: \"52c1f0a0-9858-45d7-bd19-7d705e9c4e42\") " pod="openshift-dns/node-resolver-hk4xk"
Jul 15 16:16:40.804129 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803793    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d7494c97-f5f3-47af-a6c7-7f939c53bc39-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6zrmr\" (UID: \"d7494c97-f5f3-47af-a6c7-7f939c53bc39\") " pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:40.804129 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803811    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-cni-binary-copy\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.804129 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803825    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzhbn\" (UniqueName: \"kubernetes.io/projected/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-kube-api-access-gzhbn\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.804129 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803848    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.804129 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803876    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-lib-modules\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.804129 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803908    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmb4n\" (UniqueName: \"kubernetes.io/projected/6019a7f5-5848-4017-a06d-c728a14c4f31-kube-api-access-wmb4n\") pod \"node-ca-7tktr\" (UID: \"6019a7f5-5848-4017-a06d-c728a14c4f31\") " pod="openshift-image-registry/node-ca-7tktr"
Jul 15 16:16:40.804129 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803945    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7549f9c6-5988-4771-ae95-99e674252273-dbus\") pod \"global-pull-secret-syncer-c75b9\" (UID: \"7549f9c6-5988-4771-ae95-99e674252273\") " pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:16:40.804129 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.803969    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/29f020c0-5a62-42d5-803b-99d32e49092b-host-slash\") pod \"iptables-alerter-dt445\" (UID: \"29f020c0-5a62-42d5-803b-99d32e49092b\") " pod="openshift-network-operator/iptables-alerter-dt445"
Jul 15 16:16:40.804129 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804000    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsqps\" (UniqueName: \"kubernetes.io/projected/29f020c0-5a62-42d5-803b-99d32e49092b-kube-api-access-qsqps\") pod \"iptables-alerter-dt445\" (UID: \"29f020c0-5a62-42d5-803b-99d32e49092b\") " pod="openshift-network-operator/iptables-alerter-dt445"
Jul 15 16:16:40.804129 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804022    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-etc-modprobe-d\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.804129 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804047    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-etc-sysctl-conf\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.804129 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804069    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6019a7f5-5848-4017-a06d-c728a14c4f31-serviceca\") pod \"node-ca-7tktr\" (UID: \"6019a7f5-5848-4017-a06d-c728a14c4f31\") " pod="openshift-image-registry/node-ca-7tktr"
Jul 15 16:16:40.804129 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804093    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-metrics-certs\") pod \"network-metrics-daemon-g9k24\" (UID: \"a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd\") " pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:16:40.804880 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804114    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-log-socket\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.804880 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804137    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-host-run-ovn-kubernetes\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.804880 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804167    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d7494c97-f5f3-47af-a6c7-7f939c53bc39-cnibin\") pod \"multus-additional-cni-plugins-6zrmr\" (UID: \"d7494c97-f5f3-47af-a6c7-7f939c53bc39\") " pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:40.804880 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804190    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5gn2\" (UniqueName: \"kubernetes.io/projected/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-kube-api-access-c5gn2\") pod \"network-metrics-daemon-g9k24\" (UID: \"a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd\") " pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:16:40.804880 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804215    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b4b9d03d-1841-4392-84d9-e97c648a1eec-socket-dir\") pod \"aws-ebs-csi-driver-node-tbb4b\" (UID: \"b4b9d03d-1841-4392-84d9-e97c648a1eec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b"
Jul 15 16:16:40.804880 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804238    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvn24\" (UniqueName: \"kubernetes.io/projected/9106d273-27df-4be5-84d0-c22aff403246-kube-api-access-wvn24\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.804880 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804260    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85rfm\" (UniqueName: \"kubernetes.io/projected/422a9314-d5df-4554-ae6d-2a21856dcb6c-kube-api-access-85rfm\") pod \"network-check-target-t6fc5\" (UID: \"422a9314-d5df-4554-ae6d-2a21856dcb6c\") " pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:16:40.804880 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804283    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-var-lib-openvswitch\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.804880 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804304    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-host-cni-netd\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.804880 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804326    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-etc-sysctl-d\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.804880 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804350    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-multus-cni-dir\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.804880 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804372    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-host-cni-bin\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.804880 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804394    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/46356463-bfb1-42b8-8871-71357489a95b-ovn-node-metrics-cert\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.804880 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804416    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b4b9d03d-1841-4392-84d9-e97c648a1eec-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tbb4b\" (UID: \"b4b9d03d-1841-4392-84d9-e97c648a1eec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b"
Jul 15 16:16:40.804880 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804438    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7549f9c6-5988-4771-ae95-99e674252273-original-pull-secret\") pod \"global-pull-secret-syncer-c75b9\" (UID: \"7549f9c6-5988-4771-ae95-99e674252273\") " pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:16:40.804880 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804462    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-os-release\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.805585 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804485    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-host-var-lib-kubelet\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.805585 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804517    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-systemd-units\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.805585 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804539    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-run\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.805585 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804565    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d7494c97-f5f3-47af-a6c7-7f939c53bc39-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6zrmr\" (UID: \"d7494c97-f5f3-47af-a6c7-7f939c53bc39\") " pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:40.805585 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804591    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-host-run-k8s-cni-cncf-io\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.805585 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804631    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-host-run-netns\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.805585 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804654    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-multus-daemon-config\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.805585 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804682    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/46356463-bfb1-42b8-8871-71357489a95b-ovnkube-script-lib\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.805585 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804706    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs9g2\" (UniqueName: \"kubernetes.io/projected/b4b9d03d-1841-4392-84d9-e97c648a1eec-kube-api-access-vs9g2\") pod \"aws-ebs-csi-driver-node-tbb4b\" (UID: \"b4b9d03d-1841-4392-84d9-e97c648a1eec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b"
Jul 15 16:16:40.805585 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804731    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-etc-systemd\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.805585 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804774    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d7494c97-f5f3-47af-a6c7-7f939c53bc39-os-release\") pod \"multus-additional-cni-plugins-6zrmr\" (UID: \"d7494c97-f5f3-47af-a6c7-7f939c53bc39\") " pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:40.805585 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804799    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-multus-socket-dir-parent\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.805585 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804821    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-multus-conf-dir\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.805585 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804844    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-etc-kubernetes\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.805585 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804867    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/29f020c0-5a62-42d5-803b-99d32e49092b-iptables-alerter-script\") pod \"iptables-alerter-dt445\" (UID: \"29f020c0-5a62-42d5-803b-99d32e49092b\") " pod="openshift-network-operator/iptables-alerter-dt445"
Jul 15 16:16:40.805585 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804890    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-host-slash\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.806214 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804915    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b4b9d03d-1841-4392-84d9-e97c648a1eec-registration-dir\") pod \"aws-ebs-csi-driver-node-tbb4b\" (UID: \"b4b9d03d-1841-4392-84d9-e97c648a1eec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b"
Jul 15 16:16:40.806214 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804951    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqnzq\" (UniqueName: \"kubernetes.io/projected/d7494c97-f5f3-47af-a6c7-7f939c53bc39-kube-api-access-qqnzq\") pod \"multus-additional-cni-plugins-6zrmr\" (UID: \"d7494c97-f5f3-47af-a6c7-7f939c53bc39\") " pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:40.806214 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.804987    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-host-var-lib-cni-bin\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.806214 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.805026    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-run-openvswitch\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.806214 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.805065    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-run-ovn\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.806214 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.805113    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/46356463-bfb1-42b8-8871-71357489a95b-env-overrides\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.806214 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.805139    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/79a480c8-f8ad-4d35-aac1-7753e9895ca6-agent-certs\") pod \"konnectivity-agent-gzwh7\" (UID: \"79a480c8-f8ad-4d35-aac1-7753e9895ca6\") " pod="kube-system/konnectivity-agent-gzwh7"
Jul 15 16:16:40.806214 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.805173    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b4b9d03d-1841-4392-84d9-e97c648a1eec-device-dir\") pod \"aws-ebs-csi-driver-node-tbb4b\" (UID: \"b4b9d03d-1841-4392-84d9-e97c648a1eec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b"
Jul 15 16:16:40.806214 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.805198    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b4b9d03d-1841-4392-84d9-e97c648a1eec-sys-fs\") pod \"aws-ebs-csi-driver-node-tbb4b\" (UID: \"b4b9d03d-1841-4392-84d9-e97c648a1eec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b"
Jul 15 16:16:40.806214 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.805226    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-sys\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.806214 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.805250    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52c1f0a0-9858-45d7-bd19-7d705e9c4e42-tmp-dir\") pod \"node-resolver-hk4xk\" (UID: \"52c1f0a0-9858-45d7-bd19-7d705e9c4e42\") " pod="openshift-dns/node-resolver-hk4xk"
Jul 15 16:16:40.806214 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.805278    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-node-log\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.806214 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.805304    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/79a480c8-f8ad-4d35-aac1-7753e9895ca6-konnectivity-ca\") pod \"konnectivity-agent-gzwh7\" (UID: \"79a480c8-f8ad-4d35-aac1-7753e9895ca6\") " pod="kube-system/konnectivity-agent-gzwh7"
Jul 15 16:16:40.859869 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.859838    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7n7zg"
Jul 15 16:16:40.877509 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.877455    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-132.ec2.internal" event={"ID":"dc2084eaf308dac38296e58e829ee36f","Type":"ContainerStarted","Data":"5a0b0798f574a6d95887a44d1c92a46e8744f19266dae4065830f7e3c65050f4"}
Jul 15 16:16:40.878617 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.878592    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-132.ec2.internal" event={"ID":"4c8d0721ab10fafc7cda1fc3cfef7c2b","Type":"ContainerStarted","Data":"df643003e2e7d2ad63e64443937996b74ea8c37afdb5fac9da9fc586df102728"}
Jul 15 16:16:40.883308 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.883284    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7n7zg"
Jul 15 16:16:40.906219 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906191    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-cni-binary-copy\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.906337 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzhbn\" (UniqueName: \"kubernetes.io/projected/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-kube-api-access-gzhbn\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.906337 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906258    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.906337 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906282    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-lib-modules\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.906337 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906306    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmb4n\" (UniqueName: \"kubernetes.io/projected/6019a7f5-5848-4017-a06d-c728a14c4f31-kube-api-access-wmb4n\") pod \"node-ca-7tktr\" (UID: \"6019a7f5-5848-4017-a06d-c728a14c4f31\") " pod="openshift-image-registry/node-ca-7tktr"
Jul 15 16:16:40.906337 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906328    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7549f9c6-5988-4771-ae95-99e674252273-dbus\") pod \"global-pull-secret-syncer-c75b9\" (UID: \"7549f9c6-5988-4771-ae95-99e674252273\") " pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:16:40.906565 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906369    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.906565 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/29f020c0-5a62-42d5-803b-99d32e49092b-host-slash\") pod \"iptables-alerter-dt445\" (UID: \"29f020c0-5a62-42d5-803b-99d32e49092b\") " pod="openshift-network-operator/iptables-alerter-dt445"
Jul 15 16:16:40.906565 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906445    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-lib-modules\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.906565 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906449    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsqps\" (UniqueName: \"kubernetes.io/projected/29f020c0-5a62-42d5-803b-99d32e49092b-kube-api-access-qsqps\") pod \"iptables-alerter-dt445\" (UID: \"29f020c0-5a62-42d5-803b-99d32e49092b\") " pod="openshift-network-operator/iptables-alerter-dt445"
Jul 15 16:16:40.906565 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906483    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-etc-modprobe-d\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.906565 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906496    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/29f020c0-5a62-42d5-803b-99d32e49092b-host-slash\") pod \"iptables-alerter-dt445\" (UID: \"29f020c0-5a62-42d5-803b-99d32e49092b\") " pod="openshift-network-operator/iptables-alerter-dt445"
Jul 15 16:16:40.906565 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-etc-sysctl-conf\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.906565 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906538    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6019a7f5-5848-4017-a06d-c728a14c4f31-serviceca\") pod \"node-ca-7tktr\" (UID: \"6019a7f5-5848-4017-a06d-c728a14c4f31\") " pod="openshift-image-registry/node-ca-7tktr"
Jul 15 16:16:40.906565 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906549    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7549f9c6-5988-4771-ae95-99e674252273-dbus\") pod \"global-pull-secret-syncer-c75b9\" (UID: \"7549f9c6-5988-4771-ae95-99e674252273\") " pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:16:40.906565 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906561    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-metrics-certs\") pod \"network-metrics-daemon-g9k24\" (UID: \"a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd\") " pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:16:40.907024 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906585    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-log-socket\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.907024 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906609    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-host-run-ovn-kubernetes\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.907024 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906636    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d7494c97-f5f3-47af-a6c7-7f939c53bc39-cnibin\") pod \"multus-additional-cni-plugins-6zrmr\" (UID: \"d7494c97-f5f3-47af-a6c7-7f939c53bc39\") " pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:40.907024 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906644    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-etc-modprobe-d\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.907024 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906660    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5gn2\" (UniqueName: \"kubernetes.io/projected/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-kube-api-access-c5gn2\") pod \"network-metrics-daemon-g9k24\" (UID: \"a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd\") " pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:16:40.907024 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906714    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b4b9d03d-1841-4392-84d9-e97c648a1eec-socket-dir\") pod \"aws-ebs-csi-driver-node-tbb4b\" (UID: \"b4b9d03d-1841-4392-84d9-e97c648a1eec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b"
Jul 15 16:16:40.907024 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906739    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvn24\" (UniqueName: \"kubernetes.io/projected/9106d273-27df-4be5-84d0-c22aff403246-kube-api-access-wvn24\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.907024 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906777    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85rfm\" (UniqueName: \"kubernetes.io/projected/422a9314-d5df-4554-ae6d-2a21856dcb6c-kube-api-access-85rfm\") pod \"network-check-target-t6fc5\" (UID: \"422a9314-d5df-4554-ae6d-2a21856dcb6c\") " pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:16:40.907024 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906802    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-var-lib-openvswitch\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.907024 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906826    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-log-socket\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.907024 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906826    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-host-cni-netd\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.907024 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906874    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-etc-sysctl-d\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.907024 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906870    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-host-run-ovn-kubernetes\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.907024 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906900    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-multus-cni-dir\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.907024 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906922    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b4b9d03d-1841-4392-84d9-e97c648a1eec-socket-dir\") pod \"aws-ebs-csi-driver-node-tbb4b\" (UID: \"b4b9d03d-1841-4392-84d9-e97c648a1eec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b"
Jul 15 16:16:40.907024 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906925    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-host-cni-bin\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.907024 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906963    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-host-cni-bin\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.907912 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906975    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/46356463-bfb1-42b8-8871-71357489a95b-ovn-node-metrics-cert\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.907912 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:40.906982    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:16:40.907912 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907002    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d7494c97-f5f3-47af-a6c7-7f939c53bc39-cnibin\") pod \"multus-additional-cni-plugins-6zrmr\" (UID: \"d7494c97-f5f3-47af-a6c7-7f939c53bc39\") " pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:40.907912 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907004    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b4b9d03d-1841-4392-84d9-e97c648a1eec-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tbb4b\" (UID: \"b4b9d03d-1841-4392-84d9-e97c648a1eec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b"
Jul 15 16:16:40.907912 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907050    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b4b9d03d-1841-4392-84d9-e97c648a1eec-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tbb4b\" (UID: \"b4b9d03d-1841-4392-84d9-e97c648a1eec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b"
Jul 15 16:16:40.907912 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:40.907059    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-metrics-certs podName:a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd nodeName:}" failed. No retries permitted until 2026-07-15 16:16:41.407037765 +0000 UTC m=+3.054078301 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-metrics-certs") pod "network-metrics-daemon-g9k24" (UID: "a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:16:40.907912 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907039    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7549f9c6-5988-4771-ae95-99e674252273-original-pull-secret\") pod \"global-pull-secret-syncer-c75b9\" (UID: \"7549f9c6-5988-4771-ae95-99e674252273\") " pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:16:40.907912 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-os-release\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.907912 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:40.907104    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 16:16:40.907912 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907114    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-host-var-lib-kubelet\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.907912 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907137    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-systemd-units\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.907912 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906874    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-host-cni-netd\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.907912 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:40.907174    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7549f9c6-5988-4771-ae95-99e674252273-original-pull-secret podName:7549f9c6-5988-4771-ae95-99e674252273 nodeName:}" failed. No retries permitted until 2026-07-15 16:16:41.407156238 +0000 UTC m=+3.054196758 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7549f9c6-5988-4771-ae95-99e674252273-original-pull-secret") pod "global-pull-secret-syncer-c75b9" (UID: "7549f9c6-5988-4771-ae95-99e674252273") : object "kube-system"/"original-pull-secret" not registered
Jul 15 16:16:40.907912 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907193    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6019a7f5-5848-4017-a06d-c728a14c4f31-serviceca\") pod \"node-ca-7tktr\" (UID: \"6019a7f5-5848-4017-a06d-c728a14c4f31\") " pod="openshift-image-registry/node-ca-7tktr"
Jul 15 16:16:40.907912 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907203    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-os-release\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.907912 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907213    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-host-var-lib-kubelet\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.907912 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907227    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-multus-cni-dir\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.908493 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907230    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-run\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.908493 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906779    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-etc-sysctl-conf\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.908493 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907254    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-systemd-units\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.908493 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.906777    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-cni-binary-copy\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.908493 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907276    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-var-lib-openvswitch\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.908493 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907275    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d7494c97-f5f3-47af-a6c7-7f939c53bc39-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6zrmr\" (UID: \"d7494c97-f5f3-47af-a6c7-7f939c53bc39\") " pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:40.908493 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907329    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-run\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.908493 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907364    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-etc-sysctl-d\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.908493 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907450    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-host-run-k8s-cni-cncf-io\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.908493 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907500    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-host-run-netns\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.908493 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907512    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-host-run-k8s-cni-cncf-io\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.908493 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907546    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-host-run-netns\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.908493 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907557    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-multus-daemon-config\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.908493 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907574    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/46356463-bfb1-42b8-8871-71357489a95b-ovnkube-script-lib\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.908493 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907592    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs9g2\" (UniqueName: \"kubernetes.io/projected/b4b9d03d-1841-4392-84d9-e97c648a1eec-kube-api-access-vs9g2\") pod \"aws-ebs-csi-driver-node-tbb4b\" (UID: \"b4b9d03d-1841-4392-84d9-e97c648a1eec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b"
Jul 15 16:16:40.908493 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907608    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-etc-systemd\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.908493 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907632    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d7494c97-f5f3-47af-a6c7-7f939c53bc39-os-release\") pod \"multus-additional-cni-plugins-6zrmr\" (UID: \"d7494c97-f5f3-47af-a6c7-7f939c53bc39\") " pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:40.909298 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907654    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-multus-socket-dir-parent\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.909298 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907716    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-etc-systemd\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.909298 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907788    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-multus-socket-dir-parent\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.909298 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907641    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 15 16:16:40.909298 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907852    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d7494c97-f5f3-47af-a6c7-7f939c53bc39-os-release\") pod \"multus-additional-cni-plugins-6zrmr\" (UID: \"d7494c97-f5f3-47af-a6c7-7f939c53bc39\") " pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:40.909298 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907947    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d7494c97-f5f3-47af-a6c7-7f939c53bc39-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6zrmr\" (UID: \"d7494c97-f5f3-47af-a6c7-7f939c53bc39\") " pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:40.909298 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.907965    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-multus-conf-dir\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.909298 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908033    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-multus-daemon-config\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.909298 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908317    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-multus-conf-dir\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.909298 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908360    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-etc-kubernetes\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.909298 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908385    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/29f020c0-5a62-42d5-803b-99d32e49092b-iptables-alerter-script\") pod \"iptables-alerter-dt445\" (UID: \"29f020c0-5a62-42d5-803b-99d32e49092b\") " pod="openshift-network-operator/iptables-alerter-dt445"
Jul 15 16:16:40.909298 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908407    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-host-slash\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.909298 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908409    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-etc-kubernetes\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.909298 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908430    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b4b9d03d-1841-4392-84d9-e97c648a1eec-registration-dir\") pod \"aws-ebs-csi-driver-node-tbb4b\" (UID: \"b4b9d03d-1841-4392-84d9-e97c648a1eec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b"
Jul 15 16:16:40.909298 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908455    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqnzq\" (UniqueName: \"kubernetes.io/projected/d7494c97-f5f3-47af-a6c7-7f939c53bc39-kube-api-access-qqnzq\") pod \"multus-additional-cni-plugins-6zrmr\" (UID: \"d7494c97-f5f3-47af-a6c7-7f939c53bc39\") " pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:40.909298 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908482    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-host-var-lib-cni-bin\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.909298 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908490    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-host-slash\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.909298 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908504    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-run-openvswitch\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.910161 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908509    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b4b9d03d-1841-4392-84d9-e97c648a1eec-registration-dir\") pod \"aws-ebs-csi-driver-node-tbb4b\" (UID: \"b4b9d03d-1841-4392-84d9-e97c648a1eec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b"
Jul 15 16:16:40.910161 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908530    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-run-ovn\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.910161 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908553    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/46356463-bfb1-42b8-8871-71357489a95b-env-overrides\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.910161 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908576    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/79a480c8-f8ad-4d35-aac1-7753e9895ca6-agent-certs\") pod \"konnectivity-agent-gzwh7\" (UID: \"79a480c8-f8ad-4d35-aac1-7753e9895ca6\") " pod="kube-system/konnectivity-agent-gzwh7"
Jul 15 16:16:40.910161 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908589    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-run-openvswitch\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.910161 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908597    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b4b9d03d-1841-4392-84d9-e97c648a1eec-device-dir\") pod \"aws-ebs-csi-driver-node-tbb4b\" (UID: \"b4b9d03d-1841-4392-84d9-e97c648a1eec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b"
Jul 15 16:16:40.910161 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908620    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b4b9d03d-1841-4392-84d9-e97c648a1eec-sys-fs\") pod \"aws-ebs-csi-driver-node-tbb4b\" (UID: \"b4b9d03d-1841-4392-84d9-e97c648a1eec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b"
Jul 15 16:16:40.910161 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908629    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-host-var-lib-cni-bin\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.910161 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908640    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-sys\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.910161 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52c1f0a0-9858-45d7-bd19-7d705e9c4e42-tmp-dir\") pod \"node-resolver-hk4xk\" (UID: \"52c1f0a0-9858-45d7-bd19-7d705e9c4e42\") " pod="openshift-dns/node-resolver-hk4xk"
Jul 15 16:16:40.910161 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908687    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-node-log\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.910161 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908712    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/79a480c8-f8ad-4d35-aac1-7753e9895ca6-konnectivity-ca\") pod \"konnectivity-agent-gzwh7\" (UID: \"79a480c8-f8ad-4d35-aac1-7753e9895ca6\") " pod="kube-system/konnectivity-agent-gzwh7"
Jul 15 16:16:40.910161 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908726    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-run-ovn\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.910161 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908736    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-etc-kubernetes\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.910161 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908775    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-var-lib-kubelet\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.910161 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908799    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6019a7f5-5848-4017-a06d-c728a14c4f31-host\") pod \"node-ca-7tktr\" (UID: \"6019a7f5-5848-4017-a06d-c728a14c4f31\") " pod="openshift-image-registry/node-ca-7tktr"
Jul 15 16:16:40.910161 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908840    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d7494c97-f5f3-47af-a6c7-7f939c53bc39-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6zrmr\" (UID: \"d7494c97-f5f3-47af-a6c7-7f939c53bc39\") " pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:40.910161 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908847    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/46356463-bfb1-42b8-8871-71357489a95b-ovnkube-script-lib\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.911022 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908862    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-system-cni-dir\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.911022 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908885    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-cnibin\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.911022 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908904    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-node-log\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.911022 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908913    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/29f020c0-5a62-42d5-803b-99d32e49092b-iptables-alerter-script\") pod \"iptables-alerter-dt445\" (UID: \"29f020c0-5a62-42d5-803b-99d32e49092b\") " pod="openshift-network-operator/iptables-alerter-dt445"
Jul 15 16:16:40.911022 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908945    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-host-kubelet\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.911022 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908958    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-var-lib-kubelet\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.911022 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908989    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-etc-kubernetes\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.911022 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909005    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-system-cni-dir\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.911022 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909032    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b4b9d03d-1841-4392-84d9-e97c648a1eec-device-dir\") pod \"aws-ebs-csi-driver-node-tbb4b\" (UID: \"b4b9d03d-1841-4392-84d9-e97c648a1eec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b"
Jul 15 16:16:40.911022 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909051    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-cnibin\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.911022 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.908907    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-host-kubelet\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.911022 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909071    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6019a7f5-5848-4017-a06d-c728a14c4f31-host\") pod \"node-ca-7tktr\" (UID: \"6019a7f5-5848-4017-a06d-c728a14c4f31\") " pod="openshift-image-registry/node-ca-7tktr"
Jul 15 16:16:40.911022 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909090    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-etc-openvswitch\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.911022 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909102    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-sys\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.911022 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909114    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b4b9d03d-1841-4392-84d9-e97c648a1eec-sys-fs\") pod \"aws-ebs-csi-driver-node-tbb4b\" (UID: \"b4b9d03d-1841-4392-84d9-e97c648a1eec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b"
Jul 15 16:16:40.911022 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909118    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/52c1f0a0-9858-45d7-bd19-7d705e9c4e42-hosts-file\") pod \"node-resolver-hk4xk\" (UID: \"52c1f0a0-9858-45d7-bd19-7d705e9c4e42\") " pod="openshift-dns/node-resolver-hk4xk"
Jul 15 16:16:40.911022 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909122    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/46356463-bfb1-42b8-8871-71357489a95b-env-overrides\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.911022 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909051    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d7494c97-f5f3-47af-a6c7-7f939c53bc39-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6zrmr\" (UID: \"d7494c97-f5f3-47af-a6c7-7f939c53bc39\") " pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:40.911871 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909142    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d7494c97-f5f3-47af-a6c7-7f939c53bc39-system-cni-dir\") pod \"multus-additional-cni-plugins-6zrmr\" (UID: \"d7494c97-f5f3-47af-a6c7-7f939c53bc39\") " pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:40.911871 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909166    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-hostroot\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.911871 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909177    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-etc-openvswitch\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.911871 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909185    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-host-run-netns\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.911871 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909185    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/52c1f0a0-9858-45d7-bd19-7d705e9c4e42-hosts-file\") pod \"node-resolver-hk4xk\" (UID: \"52c1f0a0-9858-45d7-bd19-7d705e9c4e42\") " pod="openshift-dns/node-resolver-hk4xk"
Jul 15 16:16:40.911871 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909220    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-hostroot\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.911871 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909220    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d7494c97-f5f3-47af-a6c7-7f939c53bc39-system-cni-dir\") pod \"multus-additional-cni-plugins-6zrmr\" (UID: \"d7494c97-f5f3-47af-a6c7-7f939c53bc39\") " pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:40.911871 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909236    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-host-run-netns\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.911871 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf4vk\" (UniqueName: \"kubernetes.io/projected/46356463-bfb1-42b8-8871-71357489a95b-kube-api-access-tf4vk\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.911871 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909290    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b4b9d03d-1841-4392-84d9-e97c648a1eec-etc-selinux\") pod \"aws-ebs-csi-driver-node-tbb4b\" (UID: \"b4b9d03d-1841-4392-84d9-e97c648a1eec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b"
Jul 15 16:16:40.911871 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909306    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52c1f0a0-9858-45d7-bd19-7d705e9c4e42-tmp-dir\") pod \"node-resolver-hk4xk\" (UID: \"52c1f0a0-9858-45d7-bd19-7d705e9c4e42\") " pod="openshift-dns/node-resolver-hk4xk"
Jul 15 16:16:40.911871 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909313    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-host-var-lib-cni-multus\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.911871 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909342    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-host-run-multus-certs\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.911871 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909364    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/46356463-bfb1-42b8-8871-71357489a95b-ovnkube-config\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.911871 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909380    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b4b9d03d-1841-4392-84d9-e97c648a1eec-etc-selinux\") pod \"aws-ebs-csi-driver-node-tbb4b\" (UID: \"b4b9d03d-1841-4392-84d9-e97c648a1eec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b"
Jul 15 16:16:40.911871 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909392    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9106d273-27df-4be5-84d0-c22aff403246-tmp\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.911871 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909417    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7549f9c6-5988-4771-ae95-99e674252273-kubelet-config\") pod \"global-pull-secret-syncer-c75b9\" (UID: \"7549f9c6-5988-4771-ae95-99e674252273\") " pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:16:40.911871 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909425    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-host-run-multus-certs\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.912712 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909392    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-host-var-lib-cni-multus\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.912712 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909438    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-host\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.912712 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909474    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d7494c97-f5f3-47af-a6c7-7f939c53bc39-cni-binary-copy\") pod \"multus-additional-cni-plugins-6zrmr\" (UID: \"d7494c97-f5f3-47af-a6c7-7f939c53bc39\") " pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:40.912712 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909483    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-host\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.912712 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909536    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7549f9c6-5988-4771-ae95-99e674252273-kubelet-config\") pod \"global-pull-secret-syncer-c75b9\" (UID: \"7549f9c6-5988-4771-ae95-99e674252273\") " pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:16:40.912712 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909589    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-run-systemd\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.912712 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909618    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-etc-sysconfig\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.912712 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909619    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/79a480c8-f8ad-4d35-aac1-7753e9895ca6-konnectivity-ca\") pod \"konnectivity-agent-gzwh7\" (UID: \"79a480c8-f8ad-4d35-aac1-7753e9895ca6\") " pod="kube-system/konnectivity-agent-gzwh7"
Jul 15 16:16:40.912712 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909655    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/46356463-bfb1-42b8-8871-71357489a95b-run-systemd\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.912712 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909688    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9106d273-27df-4be5-84d0-c22aff403246-etc-tuned\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.912712 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909716    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x66vw\" (UniqueName: \"kubernetes.io/projected/52c1f0a0-9858-45d7-bd19-7d705e9c4e42-kube-api-access-x66vw\") pod \"node-resolver-hk4xk\" (UID: \"52c1f0a0-9858-45d7-bd19-7d705e9c4e42\") " pod="openshift-dns/node-resolver-hk4xk"
Jul 15 16:16:40.912712 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909742    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d7494c97-f5f3-47af-a6c7-7f939c53bc39-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6zrmr\" (UID: \"d7494c97-f5f3-47af-a6c7-7f939c53bc39\") " pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:40.912712 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909848    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/46356463-bfb1-42b8-8871-71357489a95b-ovnkube-config\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.912712 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.909688    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9106d273-27df-4be5-84d0-c22aff403246-etc-sysconfig\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.912712 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.910159    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d7494c97-f5f3-47af-a6c7-7f939c53bc39-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6zrmr\" (UID: \"d7494c97-f5f3-47af-a6c7-7f939c53bc39\") " pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:40.912712 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.910473    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d7494c97-f5f3-47af-a6c7-7f939c53bc39-cni-binary-copy\") pod \"multus-additional-cni-plugins-6zrmr\" (UID: \"d7494c97-f5f3-47af-a6c7-7f939c53bc39\") " pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:40.912712 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.912043    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/46356463-bfb1-42b8-8871-71357489a95b-ovn-node-metrics-cert\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:40.913516 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.912322    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9106d273-27df-4be5-84d0-c22aff403246-etc-tuned\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.913516 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.912479    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/79a480c8-f8ad-4d35-aac1-7753e9895ca6-agent-certs\") pod \"konnectivity-agent-gzwh7\" (UID: \"79a480c8-f8ad-4d35-aac1-7753e9895ca6\") " pod="kube-system/konnectivity-agent-gzwh7"
Jul 15 16:16:40.913516 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.912557    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9106d273-27df-4be5-84d0-c22aff403246-tmp\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.925108 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:40.925056    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 16:16:40.925108 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:40.925080    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 16:16:40.925108 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:40.925095    2580 projected.go:196] Error preparing data for projected volume kube-api-access-85rfm for pod openshift-network-diagnostics/network-check-target-t6fc5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:16:40.925308 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:40.925143    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/422a9314-d5df-4554-ae6d-2a21856dcb6c-kube-api-access-85rfm podName:422a9314-d5df-4554-ae6d-2a21856dcb6c nodeName:}" failed. No retries permitted until 2026-07-15 16:16:41.425130415 +0000 UTC m=+3.072170934 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-85rfm" (UniqueName: "kubernetes.io/projected/422a9314-d5df-4554-ae6d-2a21856dcb6c-kube-api-access-85rfm") pod "network-check-target-t6fc5" (UID: "422a9314-d5df-4554-ae6d-2a21856dcb6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:16:40.929044 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.929020    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs9g2\" (UniqueName: \"kubernetes.io/projected/b4b9d03d-1841-4392-84d9-e97c648a1eec-kube-api-access-vs9g2\") pod \"aws-ebs-csi-driver-node-tbb4b\" (UID: \"b4b9d03d-1841-4392-84d9-e97c648a1eec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b"
Jul 15 16:16:40.936484 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.936460    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmb4n\" (UniqueName: \"kubernetes.io/projected/6019a7f5-5848-4017-a06d-c728a14c4f31-kube-api-access-wmb4n\") pod \"node-ca-7tktr\" (UID: \"6019a7f5-5848-4017-a06d-c728a14c4f31\") " pod="openshift-image-registry/node-ca-7tktr"
Jul 15 16:16:40.938272 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.938228    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqnzq\" (UniqueName: \"kubernetes.io/projected/d7494c97-f5f3-47af-a6c7-7f939c53bc39-kube-api-access-qqnzq\") pod \"multus-additional-cni-plugins-6zrmr\" (UID: \"d7494c97-f5f3-47af-a6c7-7f939c53bc39\") " pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:40.938478 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.938456    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5gn2\" (UniqueName: \"kubernetes.io/projected/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-kube-api-access-c5gn2\") pod \"network-metrics-daemon-g9k24\" (UID: \"a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd\") " pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:16:40.938586 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.938489    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsqps\" (UniqueName: \"kubernetes.io/projected/29f020c0-5a62-42d5-803b-99d32e49092b-kube-api-access-qsqps\") pod \"iptables-alerter-dt445\" (UID: \"29f020c0-5a62-42d5-803b-99d32e49092b\") " pod="openshift-network-operator/iptables-alerter-dt445"
Jul 15 16:16:40.939484 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.939461    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvn24\" (UniqueName: \"kubernetes.io/projected/9106d273-27df-4be5-84d0-c22aff403246-kube-api-access-wvn24\") pod \"tuned-vzmbb\" (UID: \"9106d273-27df-4be5-84d0-c22aff403246\") " pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:40.940912 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.940890    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzhbn\" (UniqueName: \"kubernetes.io/projected/30df3a5a-23a5-41c5-a430-7dbfe0fedbe3-kube-api-access-gzhbn\") pod \"multus-4srwh\" (UID: \"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3\") " pod="openshift-multus/multus-4srwh"
Jul 15 16:16:40.942195 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.942175    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x66vw\" (UniqueName: \"kubernetes.io/projected/52c1f0a0-9858-45d7-bd19-7d705e9c4e42-kube-api-access-x66vw\") pod \"node-resolver-hk4xk\" (UID: \"52c1f0a0-9858-45d7-bd19-7d705e9c4e42\") " pod="openshift-dns/node-resolver-hk4xk"
Jul 15 16:16:40.943596 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:40.943578    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf4vk\" (UniqueName: \"kubernetes.io/projected/46356463-bfb1-42b8-8871-71357489a95b-kube-api-access-tf4vk\") pod \"ovnkube-node-295d8\" (UID: \"46356463-bfb1-42b8-8871-71357489a95b\") " pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:41.016133 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.016099    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vzmbb"
Jul 15 16:16:41.025859 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.025828    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hk4xk"
Jul 15 16:16:41.026846 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:41.026816    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9106d273_27df_4be5_84d0_c22aff403246.slice/crio-e6b0f6cf5c36da7112a69a5d789425fc79a2476abbe3a03da4ef8ea0c470dcf7 WatchSource:0}: Error finding container e6b0f6cf5c36da7112a69a5d789425fc79a2476abbe3a03da4ef8ea0c470dcf7: Status 404 returned error can't find the container with id e6b0f6cf5c36da7112a69a5d789425fc79a2476abbe3a03da4ef8ea0c470dcf7
Jul 15 16:16:41.035566 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:41.035418    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52c1f0a0_9858_45d7_bd19_7d705e9c4e42.slice/crio-d15d05c5f8cdc2676dcd1b2318f51af773023ba5a09bb580585fc03f923689e3 WatchSource:0}: Error finding container d15d05c5f8cdc2676dcd1b2318f51af773023ba5a09bb580585fc03f923689e3: Status 404 returned error can't find the container with id d15d05c5f8cdc2676dcd1b2318f51af773023ba5a09bb580585fc03f923689e3
Jul 15 16:16:41.036886 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.036096    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7tktr"
Jul 15 16:16:41.042829 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.042797    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4srwh"
Jul 15 16:16:41.045698 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:41.045675    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6019a7f5_5848_4017_a06d_c728a14c4f31.slice/crio-66bdb8705705476313977e4fe810db1c87486564b34caa362799d00de167c2a0 WatchSource:0}: Error finding container 66bdb8705705476313977e4fe810db1c87486564b34caa362799d00de167c2a0: Status 404 returned error can't find the container with id 66bdb8705705476313977e4fe810db1c87486564b34caa362799d00de167c2a0
Jul 15 16:16:41.050737 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.050684    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6zrmr"
Jul 15 16:16:41.052401 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:41.052359    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30df3a5a_23a5_41c5_a430_7dbfe0fedbe3.slice/crio-10e262a5a9ca42d8d18161eda7ea20e07fb0f271251c347ce47a40c69176f297 WatchSource:0}: Error finding container 10e262a5a9ca42d8d18161eda7ea20e07fb0f271251c347ce47a40c69176f297: Status 404 returned error can't find the container with id 10e262a5a9ca42d8d18161eda7ea20e07fb0f271251c347ce47a40c69176f297
Jul 15 16:16:41.059675 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.059586    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dt445"
Jul 15 16:16:41.059912 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:41.059880    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7494c97_f5f3_47af_a6c7_7f939c53bc39.slice/crio-202f4e3ebd8d5ec1bfe8c28de5ef89cbb8d59bdc7362dad20fd320b0d7c7e395 WatchSource:0}: Error finding container 202f4e3ebd8d5ec1bfe8c28de5ef89cbb8d59bdc7362dad20fd320b0d7c7e395: Status 404 returned error can't find the container with id 202f4e3ebd8d5ec1bfe8c28de5ef89cbb8d59bdc7362dad20fd320b0d7c7e395
Jul 15 16:16:41.067063 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.066891    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:16:41.067063 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.067034    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 16:16:41.067581 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:41.067561    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29f020c0_5a62_42d5_803b_99d32e49092b.slice/crio-4813daf3ec2b54882885bbb0782a327e61f17146d7a071212975497e2f024716 WatchSource:0}: Error finding container 4813daf3ec2b54882885bbb0782a327e61f17146d7a071212975497e2f024716: Status 404 returned error can't find the container with id 4813daf3ec2b54882885bbb0782a327e61f17146d7a071212975497e2f024716
Jul 15 16:16:41.075451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.075434    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gzwh7"
Jul 15 16:16:41.075903 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:41.075881    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46356463_bfb1_42b8_8871_71357489a95b.slice/crio-2f715f485b526a3b2baca602d51504d2c80d5ed4a332aa0787f63b5e9c52a63a WatchSource:0}: Error finding container 2f715f485b526a3b2baca602d51504d2c80d5ed4a332aa0787f63b5e9c52a63a: Status 404 returned error can't find the container with id 2f715f485b526a3b2baca602d51504d2c80d5ed4a332aa0787f63b5e9c52a63a
Jul 15 16:16:41.080962 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.080942    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b"
Jul 15 16:16:41.084776 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:41.084738    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79a480c8_f8ad_4d35_aac1_7753e9895ca6.slice/crio-136f37d4f36c43afbf4b310662c5a22adc65b892bb86f6318991867b517ce98c WatchSource:0}: Error finding container 136f37d4f36c43afbf4b310662c5a22adc65b892bb86f6318991867b517ce98c: Status 404 returned error can't find the container with id 136f37d4f36c43afbf4b310662c5a22adc65b892bb86f6318991867b517ce98c
Jul 15 16:16:41.092180 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:16:41.092153    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4b9d03d_1841_4392_84d9_e97c648a1eec.slice/crio-b10d8b1f3597f020def4b11f37760f476a553d0aea96b610555a2291ccc31a21 WatchSource:0}: Error finding container b10d8b1f3597f020def4b11f37760f476a553d0aea96b610555a2291ccc31a21: Status 404 returned error can't find the container with id b10d8b1f3597f020def4b11f37760f476a553d0aea96b610555a2291ccc31a21
Jul 15 16:16:41.414785 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.414679    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-metrics-certs\") pod \"network-metrics-daemon-g9k24\" (UID: \"a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd\") " pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:16:41.414785 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.414744    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7549f9c6-5988-4771-ae95-99e674252273-original-pull-secret\") pod \"global-pull-secret-syncer-c75b9\" (UID: \"7549f9c6-5988-4771-ae95-99e674252273\") " pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:16:41.414996 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:41.414858    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:16:41.414996 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:41.414931    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-metrics-certs podName:a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd nodeName:}" failed. No retries permitted until 2026-07-15 16:16:42.41491104 +0000 UTC m=+4.061951557 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-metrics-certs") pod "network-metrics-daemon-g9k24" (UID: "a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:16:41.414996 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:41.414863    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 16:16:41.415132 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:41.415015    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7549f9c6-5988-4771-ae95-99e674252273-original-pull-secret podName:7549f9c6-5988-4771-ae95-99e674252273 nodeName:}" failed. No retries permitted until 2026-07-15 16:16:42.414998123 +0000 UTC m=+4.062038652 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7549f9c6-5988-4771-ae95-99e674252273-original-pull-secret") pod "global-pull-secret-syncer-c75b9" (UID: "7549f9c6-5988-4771-ae95-99e674252273") : object "kube-system"/"original-pull-secret" not registered
Jul 15 16:16:41.516028 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.515998    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85rfm\" (UniqueName: \"kubernetes.io/projected/422a9314-d5df-4554-ae6d-2a21856dcb6c-kube-api-access-85rfm\") pod \"network-check-target-t6fc5\" (UID: \"422a9314-d5df-4554-ae6d-2a21856dcb6c\") " pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:16:41.516208 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:41.516166    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 16:16:41.516208 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:41.516188    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 16:16:41.516208 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:41.516200    2580 projected.go:196] Error preparing data for projected volume kube-api-access-85rfm for pod openshift-network-diagnostics/network-check-target-t6fc5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:16:41.516340 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:41.516254    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/422a9314-d5df-4554-ae6d-2a21856dcb6c-kube-api-access-85rfm podName:422a9314-d5df-4554-ae6d-2a21856dcb6c nodeName:}" failed. No retries permitted until 2026-07-15 16:16:42.51623553 +0000 UTC m=+4.163276064 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-85rfm" (UniqueName: "kubernetes.io/projected/422a9314-d5df-4554-ae6d-2a21856dcb6c-kube-api-access-85rfm") pod "network-check-target-t6fc5" (UID: "422a9314-d5df-4554-ae6d-2a21856dcb6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:16:41.873267 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.873229    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:16:41.873813 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:41.873381    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t6fc5" podUID="422a9314-d5df-4554-ae6d-2a21856dcb6c"
Jul 15 16:16:41.873813 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.873434    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:16:41.873813 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:41.873545    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9k24" podUID="a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd"
Jul 15 16:16:41.881366 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.881342    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b" event={"ID":"b4b9d03d-1841-4392-84d9-e97c648a1eec","Type":"ContainerStarted","Data":"b10d8b1f3597f020def4b11f37760f476a553d0aea96b610555a2291ccc31a21"}
Jul 15 16:16:41.882460 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.882420    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gzwh7" event={"ID":"79a480c8-f8ad-4d35-aac1-7753e9895ca6","Type":"ContainerStarted","Data":"136f37d4f36c43afbf4b310662c5a22adc65b892bb86f6318991867b517ce98c"}
Jul 15 16:16:41.883696 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.883665    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-295d8" event={"ID":"46356463-bfb1-42b8-8871-71357489a95b","Type":"ContainerStarted","Data":"2f715f485b526a3b2baca602d51504d2c80d5ed4a332aa0787f63b5e9c52a63a"}
Jul 15 16:16:41.884817 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.884785    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dt445" event={"ID":"29f020c0-5a62-42d5-803b-99d32e49092b","Type":"ContainerStarted","Data":"4813daf3ec2b54882885bbb0782a327e61f17146d7a071212975497e2f024716"}
Jul 15 16:16:41.885060 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.885026    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 16:11:40 +0000 UTC" deadline="2028-02-05 22:34:14.098867351 +0000 UTC"
Jul 15 16:16:41.885060 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.885057    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13686h17m32.213815143s"
Jul 15 16:16:41.885890 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.885857    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7tktr" event={"ID":"6019a7f5-5848-4017-a06d-c728a14c4f31","Type":"ContainerStarted","Data":"66bdb8705705476313977e4fe810db1c87486564b34caa362799d00de167c2a0"}
Jul 15 16:16:41.886896 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.886851    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hk4xk" event={"ID":"52c1f0a0-9858-45d7-bd19-7d705e9c4e42","Type":"ContainerStarted","Data":"d15d05c5f8cdc2676dcd1b2318f51af773023ba5a09bb580585fc03f923689e3"}
Jul 15 16:16:41.887958 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.887939    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zrmr" event={"ID":"d7494c97-f5f3-47af-a6c7-7f939c53bc39","Type":"ContainerStarted","Data":"202f4e3ebd8d5ec1bfe8c28de5ef89cbb8d59bdc7362dad20fd320b0d7c7e395"}
Jul 15 16:16:41.889127 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.889107    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4srwh" event={"ID":"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3","Type":"ContainerStarted","Data":"10e262a5a9ca42d8d18161eda7ea20e07fb0f271251c347ce47a40c69176f297"}
Jul 15 16:16:41.890157 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:41.890137    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vzmbb" event={"ID":"9106d273-27df-4be5-84d0-c22aff403246","Type":"ContainerStarted","Data":"e6b0f6cf5c36da7112a69a5d789425fc79a2476abbe3a03da4ef8ea0c470dcf7"}
Jul 15 16:16:42.423808 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:42.423379    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-metrics-certs\") pod \"network-metrics-daemon-g9k24\" (UID: \"a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd\") " pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:16:42.423808 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:42.423439    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7549f9c6-5988-4771-ae95-99e674252273-original-pull-secret\") pod \"global-pull-secret-syncer-c75b9\" (UID: \"7549f9c6-5988-4771-ae95-99e674252273\") " pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:16:42.423808 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:42.423587    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 16:16:42.423808 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:42.423641    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7549f9c6-5988-4771-ae95-99e674252273-original-pull-secret podName:7549f9c6-5988-4771-ae95-99e674252273 nodeName:}" failed. No retries permitted until 2026-07-15 16:16:44.423622834 +0000 UTC m=+6.070663365 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7549f9c6-5988-4771-ae95-99e674252273-original-pull-secret") pod "global-pull-secret-syncer-c75b9" (UID: "7549f9c6-5988-4771-ae95-99e674252273") : object "kube-system"/"original-pull-secret" not registered
Jul 15 16:16:42.423808 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:42.423728    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:16:42.423808 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:42.423780    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-metrics-certs podName:a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd nodeName:}" failed. No retries permitted until 2026-07-15 16:16:44.423749698 +0000 UTC m=+6.070790213 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-metrics-certs") pod "network-metrics-daemon-g9k24" (UID: "a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:16:42.525403 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:42.524692    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85rfm\" (UniqueName: \"kubernetes.io/projected/422a9314-d5df-4554-ae6d-2a21856dcb6c-kube-api-access-85rfm\") pod \"network-check-target-t6fc5\" (UID: \"422a9314-d5df-4554-ae6d-2a21856dcb6c\") " pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:16:42.525403 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:42.524982    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 16:16:42.525403 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:42.525008    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 16:16:42.525403 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:42.525021    2580 projected.go:196] Error preparing data for projected volume kube-api-access-85rfm for pod openshift-network-diagnostics/network-check-target-t6fc5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:16:42.525403 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:42.525080    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/422a9314-d5df-4554-ae6d-2a21856dcb6c-kube-api-access-85rfm podName:422a9314-d5df-4554-ae6d-2a21856dcb6c nodeName:}" failed. No retries permitted until 2026-07-15 16:16:44.52506211 +0000 UTC m=+6.172102638 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-85rfm" (UniqueName: "kubernetes.io/projected/422a9314-d5df-4554-ae6d-2a21856dcb6c-kube-api-access-85rfm") pod "network-check-target-t6fc5" (UID: "422a9314-d5df-4554-ae6d-2a21856dcb6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:16:42.873612 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:42.873561    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:16:42.874192 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:42.873738    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75b9" podUID="7549f9c6-5988-4771-ae95-99e674252273"
Jul 15 16:16:42.898104 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:42.897866    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 16:11:40 +0000 UTC" deadline="2028-03-05 17:33:26.062809591 +0000 UTC"
Jul 15 16:16:42.898104 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:42.897921    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14377h16m43.164892945s"
Jul 15 16:16:42.922832 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:42.922543    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-132.ec2.internal" event={"ID":"dc2084eaf308dac38296e58e829ee36f","Type":"ContainerStarted","Data":"73bc041c5024b219d4f1b92ab4cab38135ffd51c9f6fad8a59231fbecf42477e"}
Jul 15 16:16:43.873966 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:43.873299    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:16:43.873966 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:43.873415    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t6fc5" podUID="422a9314-d5df-4554-ae6d-2a21856dcb6c"
Jul 15 16:16:43.873966 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:43.873813    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:16:43.873966 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:43.873919    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9k24" podUID="a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd"
Jul 15 16:16:43.932746 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:43.931465    2580 generic.go:356] "Generic (PLEG): container finished" podID="4c8d0721ab10fafc7cda1fc3cfef7c2b" containerID="5e6f671c51ffd1b3d009b15550d7bc57c9bed7bcb84b634ba342cc2ea63025d1" exitCode=0
Jul 15 16:16:43.932746 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:43.932474    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-132.ec2.internal" event={"ID":"4c8d0721ab10fafc7cda1fc3cfef7c2b","Type":"ContainerDied","Data":"5e6f671c51ffd1b3d009b15550d7bc57c9bed7bcb84b634ba342cc2ea63025d1"}
Jul 15 16:16:43.958621 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:43.957846    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-132.ec2.internal" podStartSLOduration=4.957823574 podStartE2EDuration="4.957823574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:16:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:16:42.990094175 +0000 UTC m=+4.637134713" watchObservedRunningTime="2026-07-15 16:16:43.957823574 +0000 UTC m=+5.604864110"
Jul 15 16:16:44.440428 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:44.440392    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-metrics-certs\") pod \"network-metrics-daemon-g9k24\" (UID: \"a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd\") " pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:16:44.440606 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:44.440460    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7549f9c6-5988-4771-ae95-99e674252273-original-pull-secret\") pod \"global-pull-secret-syncer-c75b9\" (UID: \"7549f9c6-5988-4771-ae95-99e674252273\") " pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:16:44.440664 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:44.440613    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 16:16:44.440716 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:44.440679    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7549f9c6-5988-4771-ae95-99e674252273-original-pull-secret podName:7549f9c6-5988-4771-ae95-99e674252273 nodeName:}" failed. No retries permitted until 2026-07-15 16:16:48.440658685 +0000 UTC m=+10.087699214 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7549f9c6-5988-4771-ae95-99e674252273-original-pull-secret") pod "global-pull-secret-syncer-c75b9" (UID: "7549f9c6-5988-4771-ae95-99e674252273") : object "kube-system"/"original-pull-secret" not registered
Jul 15 16:16:44.440982 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:44.440868    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:16:44.440982 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:44.440929    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-metrics-certs podName:a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd nodeName:}" failed. No retries permitted until 2026-07-15 16:16:48.440912726 +0000 UTC m=+10.087953244 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-metrics-certs") pod "network-metrics-daemon-g9k24" (UID: "a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:16:44.541708 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:44.541023    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85rfm\" (UniqueName: \"kubernetes.io/projected/422a9314-d5df-4554-ae6d-2a21856dcb6c-kube-api-access-85rfm\") pod \"network-check-target-t6fc5\" (UID: \"422a9314-d5df-4554-ae6d-2a21856dcb6c\") " pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:16:44.541708 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:44.541254    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 16:16:44.541708 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:44.541273    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 16:16:44.541708 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:44.541285    2580 projected.go:196] Error preparing data for projected volume kube-api-access-85rfm for pod openshift-network-diagnostics/network-check-target-t6fc5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:16:44.541708 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:44.541350    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/422a9314-d5df-4554-ae6d-2a21856dcb6c-kube-api-access-85rfm podName:422a9314-d5df-4554-ae6d-2a21856dcb6c nodeName:}" failed. No retries permitted until 2026-07-15 16:16:48.541330979 +0000 UTC m=+10.188371516 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-85rfm" (UniqueName: "kubernetes.io/projected/422a9314-d5df-4554-ae6d-2a21856dcb6c-kube-api-access-85rfm") pod "network-check-target-t6fc5" (UID: "422a9314-d5df-4554-ae6d-2a21856dcb6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:16:44.873822 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:44.873789    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:16:44.874001 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:44.873939    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75b9" podUID="7549f9c6-5988-4771-ae95-99e674252273"
Jul 15 16:16:45.873100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:45.873064    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:16:45.873289 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:45.873075    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:16:45.873289 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:45.873232    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9k24" podUID="a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd"
Jul 15 16:16:45.873289 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:45.873266    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t6fc5" podUID="422a9314-d5df-4554-ae6d-2a21856dcb6c"
Jul 15 16:16:46.873348 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:46.873232    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:16:46.873820 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:46.873369    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75b9" podUID="7549f9c6-5988-4771-ae95-99e674252273"
Jul 15 16:16:47.873221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:47.873183    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:16:47.873735 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:47.873699    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t6fc5" podUID="422a9314-d5df-4554-ae6d-2a21856dcb6c"
Jul 15 16:16:47.874166 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:47.874091    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:16:47.874234 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:47.874197    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9k24" podUID="a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd"
Jul 15 16:16:48.478183 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:48.477957    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-metrics-certs\") pod \"network-metrics-daemon-g9k24\" (UID: \"a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd\") " pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:16:48.478183 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:48.478024    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7549f9c6-5988-4771-ae95-99e674252273-original-pull-secret\") pod \"global-pull-secret-syncer-c75b9\" (UID: \"7549f9c6-5988-4771-ae95-99e674252273\") " pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:16:48.478183 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:48.478152    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 16:16:48.478473 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:48.478218    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7549f9c6-5988-4771-ae95-99e674252273-original-pull-secret podName:7549f9c6-5988-4771-ae95-99e674252273 nodeName:}" failed. No retries permitted until 2026-07-15 16:16:56.478199153 +0000 UTC m=+18.125239680 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7549f9c6-5988-4771-ae95-99e674252273-original-pull-secret") pod "global-pull-secret-syncer-c75b9" (UID: "7549f9c6-5988-4771-ae95-99e674252273") : object "kube-system"/"original-pull-secret" not registered
Jul 15 16:16:48.478473 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:48.478284    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:16:48.478473 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:48.478318    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-metrics-certs podName:a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd nodeName:}" failed. No retries permitted until 2026-07-15 16:16:56.478306662 +0000 UTC m=+18.125347190 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-metrics-certs") pod "network-metrics-daemon-g9k24" (UID: "a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:16:48.578570 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:48.578527    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85rfm\" (UniqueName: \"kubernetes.io/projected/422a9314-d5df-4554-ae6d-2a21856dcb6c-kube-api-access-85rfm\") pod \"network-check-target-t6fc5\" (UID: \"422a9314-d5df-4554-ae6d-2a21856dcb6c\") " pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:16:48.578746 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:48.578718    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 16:16:48.578815 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:48.578749    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 16:16:48.578815 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:48.578780    2580 projected.go:196] Error preparing data for projected volume kube-api-access-85rfm for pod openshift-network-diagnostics/network-check-target-t6fc5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:16:48.578916 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:48.578846    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/422a9314-d5df-4554-ae6d-2a21856dcb6c-kube-api-access-85rfm podName:422a9314-d5df-4554-ae6d-2a21856dcb6c nodeName:}" failed. No retries permitted until 2026-07-15 16:16:56.578827304 +0000 UTC m=+18.225867832 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-85rfm" (UniqueName: "kubernetes.io/projected/422a9314-d5df-4554-ae6d-2a21856dcb6c-kube-api-access-85rfm") pod "network-check-target-t6fc5" (UID: "422a9314-d5df-4554-ae6d-2a21856dcb6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:16:48.874450 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:48.874410    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:16:48.874908 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:48.874538    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75b9" podUID="7549f9c6-5988-4771-ae95-99e674252273"
Jul 15 16:16:49.873747 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:49.873649    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:16:49.873747 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:49.873675    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:16:49.874104 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:49.873802    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t6fc5" podUID="422a9314-d5df-4554-ae6d-2a21856dcb6c"
Jul 15 16:16:49.874104 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:49.873948    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9k24" podUID="a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd"
Jul 15 16:16:50.873455 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:50.873425    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:16:50.873882 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:50.873549    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75b9" podUID="7549f9c6-5988-4771-ae95-99e674252273"
Jul 15 16:16:51.873567 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:51.873533    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:16:51.874069 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:51.873681    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9k24" podUID="a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd"
Jul 15 16:16:51.874069 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:51.873736    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:16:51.874069 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:51.873876    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t6fc5" podUID="422a9314-d5df-4554-ae6d-2a21856dcb6c"
Jul 15 16:16:52.873584 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:52.873552    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:16:52.874038 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:52.873696    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75b9" podUID="7549f9c6-5988-4771-ae95-99e674252273"
Jul 15 16:16:53.873247 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:53.873207    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:16:53.873449 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:53.873270    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:16:53.873449 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:53.873348    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9k24" podUID="a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd"
Jul 15 16:16:53.873449 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:53.873427    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t6fc5" podUID="422a9314-d5df-4554-ae6d-2a21856dcb6c"
Jul 15 16:16:54.873733 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:54.873691    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:16:54.874198 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:54.873843    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75b9" podUID="7549f9c6-5988-4771-ae95-99e674252273"
Jul 15 16:16:55.873498 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:55.873462    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:16:55.873740 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:55.873462    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:16:55.873740 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:55.873621    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9k24" podUID="a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd"
Jul 15 16:16:55.873740 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:55.873684    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t6fc5" podUID="422a9314-d5df-4554-ae6d-2a21856dcb6c"
Jul 15 16:16:56.539520 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:56.539479    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-metrics-certs\") pod \"network-metrics-daemon-g9k24\" (UID: \"a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd\") " pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:16:56.539745 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:56.539552    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7549f9c6-5988-4771-ae95-99e674252273-original-pull-secret\") pod \"global-pull-secret-syncer-c75b9\" (UID: \"7549f9c6-5988-4771-ae95-99e674252273\") " pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:16:56.539745 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:56.539676    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 16:16:56.539745 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:56.539739    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7549f9c6-5988-4771-ae95-99e674252273-original-pull-secret podName:7549f9c6-5988-4771-ae95-99e674252273 nodeName:}" failed. No retries permitted until 2026-07-15 16:17:12.53971934 +0000 UTC m=+34.186759861 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7549f9c6-5988-4771-ae95-99e674252273-original-pull-secret") pod "global-pull-secret-syncer-c75b9" (UID: "7549f9c6-5988-4771-ae95-99e674252273") : object "kube-system"/"original-pull-secret" not registered
Jul 15 16:16:56.539938 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:56.539673    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:16:56.539938 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:56.539826    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-metrics-certs podName:a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd nodeName:}" failed. No retries permitted until 2026-07-15 16:17:12.539806832 +0000 UTC m=+34.186847346 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-metrics-certs") pod "network-metrics-daemon-g9k24" (UID: "a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:16:56.640386 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:56.640353    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85rfm\" (UniqueName: \"kubernetes.io/projected/422a9314-d5df-4554-ae6d-2a21856dcb6c-kube-api-access-85rfm\") pod \"network-check-target-t6fc5\" (UID: \"422a9314-d5df-4554-ae6d-2a21856dcb6c\") " pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:16:56.640560 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:56.640510    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 16:16:56.640560 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:56.640530    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 16:16:56.640560 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:56.640543    2580 projected.go:196] Error preparing data for projected volume kube-api-access-85rfm for pod openshift-network-diagnostics/network-check-target-t6fc5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:16:56.640676 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:56.640604    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/422a9314-d5df-4554-ae6d-2a21856dcb6c-kube-api-access-85rfm podName:422a9314-d5df-4554-ae6d-2a21856dcb6c nodeName:}" failed. No retries permitted until 2026-07-15 16:17:12.640587908 +0000 UTC m=+34.287628429 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-85rfm" (UniqueName: "kubernetes.io/projected/422a9314-d5df-4554-ae6d-2a21856dcb6c-kube-api-access-85rfm") pod "network-check-target-t6fc5" (UID: "422a9314-d5df-4554-ae6d-2a21856dcb6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:16:56.873728 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:56.873686    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:16:56.873924 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:56.873878    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75b9" podUID="7549f9c6-5988-4771-ae95-99e674252273"
Jul 15 16:16:57.873047 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:57.873007    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:16:57.873243 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:57.873039    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:16:57.873243 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:57.873139    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t6fc5" podUID="422a9314-d5df-4554-ae6d-2a21856dcb6c"
Jul 15 16:16:57.873362 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:57.873266    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9k24" podUID="a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd"
Jul 15 16:16:58.874567 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:58.874532    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:16:58.875081 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:58.874660    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75b9" podUID="7549f9c6-5988-4771-ae95-99e674252273"
Jul 15 16:16:59.873398 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:59.873367    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:16:59.873587 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:16:59.873366    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:16:59.873587 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:59.873493    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9k24" podUID="a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd"
Jul 15 16:16:59.873587 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:16:59.873558    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t6fc5" podUID="422a9314-d5df-4554-ae6d-2a21856dcb6c"
Jul 15 16:17:00.874329 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:00.874045    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:17:00.874329 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:00.874180    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75b9" podUID="7549f9c6-5988-4771-ae95-99e674252273"
Jul 15 16:17:00.966903 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:00.966875    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-132.ec2.internal" event={"ID":"4c8d0721ab10fafc7cda1fc3cfef7c2b","Type":"ContainerStarted","Data":"9fe6d16b87be530a113047162dca89589f5d9a011c8e1ad6070a6e7e213743fd"}
Jul 15 16:17:00.968665 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:00.968636    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zrmr" event={"ID":"d7494c97-f5f3-47af-a6c7-7f939c53bc39","Type":"ContainerStarted","Data":"34e927f7ec4591898109a6eaf8c1bba04193054836f1ef9bd3f2d0dff76994e5"}
Jul 15 16:17:00.970322 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:00.970250    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4srwh" event={"ID":"30df3a5a-23a5-41c5-a430-7dbfe0fedbe3","Type":"ContainerStarted","Data":"41ab550ed3dda47c71e24b527c2151bca26649138e80cbaa8d8fa3ee6883686e"}
Jul 15 16:17:00.972262 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:00.972238    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vzmbb" event={"ID":"9106d273-27df-4be5-84d0-c22aff403246","Type":"ContainerStarted","Data":"d1feb3b5ccc39396a62a7b756b128f61df76af78c96c2f7602e1cc339150e828"}
Jul 15 16:17:00.974175 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:00.974153    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b" event={"ID":"b4b9d03d-1841-4392-84d9-e97c648a1eec","Type":"ContainerStarted","Data":"cec3e88abe720f0da763ecb41ab12263ad43ef748a740bb55ccb55ac40985a03"}
Jul 15 16:17:00.976077 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:00.976039    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gzwh7" event={"ID":"79a480c8-f8ad-4d35-aac1-7753e9895ca6","Type":"ContainerStarted","Data":"7fc55eea6c0df452bc11756e53fb4442ec5b10817bee947fddb273c7f8a8b097"}
Jul 15 16:17:00.977663 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:00.977627    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-295d8" event={"ID":"46356463-bfb1-42b8-8871-71357489a95b","Type":"ContainerStarted","Data":"42ac07cfd22db2377772714370671274e6fd44163b189caa2c0bdb4d2253b0ff"}
Jul 15 16:17:00.978982 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:00.978965    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7tktr" event={"ID":"6019a7f5-5848-4017-a06d-c728a14c4f31","Type":"ContainerStarted","Data":"88de6b1b8b6720e2a0db9ea0b8cad9777f43fbba1f84037e43ddb70d22b4dfb2"}
Jul 15 16:17:00.982496 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:00.982473    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hk4xk" event={"ID":"52c1f0a0-9858-45d7-bd19-7d705e9c4e42","Type":"ContainerStarted","Data":"549e33695d30e16cd3492bdf92c1501020435f8a5b9ebca635cae350c8e94cce"}
Jul 15 16:17:01.032637 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:01.032582    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-132.ec2.internal" podStartSLOduration=22.032561747 podStartE2EDuration="22.032561747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:16:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:17:01.028887496 +0000 UTC m=+22.675928032" watchObservedRunningTime="2026-07-15 16:17:01.032561747 +0000 UTC m=+22.679602283"
Jul 15 16:17:01.140375 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:01.140330    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7tktr" podStartSLOduration=2.543144996 podStartE2EDuration="22.140315251s" totalImagesPullingTime="19.597170255s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:17:01.071889663 +0000 UTC m=+22.718930224" watchObservedRunningTime="2026-07-15 16:17:01.140315251 +0000 UTC m=+22.787355787"
Jul 15 16:17:01.249974 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:01.249933    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vzmbb" podStartSLOduration=2.57983879 podStartE2EDuration="22.249918288s" totalImagesPullingTime="19.670079498s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:17:01.194702327 +0000 UTC m=+22.841742863" watchObservedRunningTime="2026-07-15 16:17:01.249918288 +0000 UTC m=+22.896958823"
Jul 15 16:17:01.295654 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:01.295614    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hk4xk" podStartSLOduration=2.6351842149999998 podStartE2EDuration="22.295598796s" totalImagesPullingTime="19.660414581s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:17:01.251325445 +0000 UTC m=+22.898365982" watchObservedRunningTime="2026-07-15 16:17:01.295598796 +0000 UTC m=+22.942639333"
Jul 15 16:17:01.338476 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:01.338420    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4srwh" podStartSLOduration=2.67184004 podStartE2EDuration="22.338398299s" totalImagesPullingTime="19.666558259s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:17:01.296101375 +0000 UTC m=+22.943141935" watchObservedRunningTime="2026-07-15 16:17:01.338398299 +0000 UTC m=+22.985438835"
Jul 15 16:17:01.364480 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:01.364449    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:01.613293 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:01.612992    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:01.872884 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:01.872861    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:17:01.873054 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:01.872986    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t6fc5" podUID="422a9314-d5df-4554-ae6d-2a21856dcb6c"
Jul 15 16:17:01.873054 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:01.873004    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:17:01.873155 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:01.873132    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9k24" podUID="a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd"
Jul 15 16:17:01.873285 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:01.873268    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:01.985237 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:01.985197    2580 generic.go:356] "Generic (PLEG): container finished" podID="d7494c97-f5f3-47af-a6c7-7f939c53bc39" containerID="34e927f7ec4591898109a6eaf8c1bba04193054836f1ef9bd3f2d0dff76994e5" exitCode=0
Jul 15 16:17:01.985237 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:01.985229    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zrmr" event={"ID":"d7494c97-f5f3-47af-a6c7-7f939c53bc39","Type":"ContainerDied","Data":"34e927f7ec4591898109a6eaf8c1bba04193054836f1ef9bd3f2d0dff76994e5"}
Jul 15 16:17:01.988199 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:01.988184    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-295d8_46356463-bfb1-42b8-8871-71357489a95b/ovn-acl-logging/0.log"
Jul 15 16:17:01.988513 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:01.988496    2580 generic.go:356] "Generic (PLEG): container finished" podID="46356463-bfb1-42b8-8871-71357489a95b" containerID="bb926a84c9b5b56dc62cdbe0efec377f607377e29006050a44280b93457e4c22" exitCode=1
Jul 15 16:17:01.988577 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:01.988558    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-295d8" event={"ID":"46356463-bfb1-42b8-8871-71357489a95b","Type":"ContainerStarted","Data":"4c6b564cd03cef5a42732590cd57bd751a72da9f011801f610b40d323abf8fb9"}
Jul 15 16:17:01.988642 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:01.988629    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-295d8" event={"ID":"46356463-bfb1-42b8-8871-71357489a95b","Type":"ContainerStarted","Data":"a252f0a4f7c77efef8a07e3b2473f4333a7822a427fd8f2aa5a047222619440d"}
Jul 15 16:17:01.988693 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:01.988681    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-295d8" event={"ID":"46356463-bfb1-42b8-8871-71357489a95b","Type":"ContainerStarted","Data":"b5c48f1aa279da95a5f167601b8d1e0b0898fd221ad2d57514622398f9f6c344"}
Jul 15 16:17:01.988729 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:01.988714    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-295d8" event={"ID":"46356463-bfb1-42b8-8871-71357489a95b","Type":"ContainerStarted","Data":"42085b23811f18af36c33c56acae6246b17d84faf43127a276c42b451a844f8b"}
Jul 15 16:17:01.988793 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:01.988772    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-295d8" event={"ID":"46356463-bfb1-42b8-8871-71357489a95b","Type":"ContainerDied","Data":"bb926a84c9b5b56dc62cdbe0efec377f607377e29006050a44280b93457e4c22"}
Jul 15 16:17:01.989932 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:01.989907    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dt445" event={"ID":"29f020c0-5a62-42d5-803b-99d32e49092b","Type":"ContainerStarted","Data":"5f7073cc0cb95952d56d616cd121192ff8497c48c1a8c20dc581415283d5d46a"}
Jul 15 16:17:02.029633 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:02.029583    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gzwh7" podStartSLOduration=3.41887277 podStartE2EDuration="23.02956699s" totalImagesPullingTime="19.61069422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:17:01.339295589 +0000 UTC m=+22.986336139" watchObservedRunningTime="2026-07-15 16:17:02.02956699 +0000 UTC m=+23.676607528"
Jul 15 16:17:02.059448 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:02.059409    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dt445" podStartSLOduration=8.023867271 podStartE2EDuration="23.059394253s" totalImagesPullingTime="15.035526982s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:17:02.058890157 +0000 UTC m=+23.705930706" watchObservedRunningTime="2026-07-15 16:17:02.059394253 +0000 UTC m=+23.706434780"
Jul 15 16:17:02.102145 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:02.102122    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 15 16:17:02.510281 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:02.510247    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:02.760606 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:02.760449    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T16:17:02.102139565Z","UUID":"2e50845e-a23c-4f55-9470-43ad9e4fa028","Handler":null,"Name":"","Endpoint":""}
Jul 15 16:17:02.763559 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:02.763537    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 15 16:17:02.763703 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:02.763581    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 15 16:17:02.863974 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:02.863937    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:02.877104 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:02.877080    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:17:02.877260 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:02.877195    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75b9" podUID="7549f9c6-5988-4771-ae95-99e674252273"
Jul 15 16:17:02.995743 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:02.995683    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b" event={"ID":"b4b9d03d-1841-4392-84d9-e97c648a1eec","Type":"ContainerStarted","Data":"26c0de707066e429efee32014fd71e91b7d90d7fddd5d03ae2dda9bfaa3e9ef5"}
Jul 15 16:17:03.020172 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:03.020100    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:03.278873 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:03.278779    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:03.466807 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:03.466770    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:03.592909 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:03.592880    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gzwh7"
Jul 15 16:17:03.593526 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:03.593502    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gzwh7"
Jul 15 16:17:03.694332 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:03.694263    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:03.874098 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:03.873849    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:17:03.874263 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:03.873851    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:17:03.874263 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:03.874243    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9k24" podUID="a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd"
Jul 15 16:17:03.874376 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:03.874324    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t6fc5" podUID="422a9314-d5df-4554-ae6d-2a21856dcb6c"
Jul 15 16:17:03.878716 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:03.878690    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:04.001457 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:04.001423    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-295d8_46356463-bfb1-42b8-8871-71357489a95b/ovn-acl-logging/0.log"
Jul 15 16:17:04.001922 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:04.001894    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-295d8" event={"ID":"46356463-bfb1-42b8-8871-71357489a95b","Type":"ContainerStarted","Data":"fa7c0599ef2e847e44c500e14dc4a3a0cbd45a76e5d236671482ccc5f8875473"}
Jul 15 16:17:04.004098 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:04.004066    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b" event={"ID":"b4b9d03d-1841-4392-84d9-e97c648a1eec","Type":"ContainerStarted","Data":"eb5dbd71607ba118b280c807160d903a9b223c22e8f22fff1a84ac114c80ba3c"}
Jul 15 16:17:04.036641 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:04.036581    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbb4b" podStartSLOduration=2.738622459 podStartE2EDuration="25.036563646s" totalImagesPullingTime="22.297941187s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:16:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:17:04.036203148 +0000 UTC m=+25.683243686" watchObservedRunningTime="2026-07-15 16:17:04.036563646 +0000 UTC m=+25.683604181"
Jul 15 16:17:04.475413 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:04.475377    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:04.679099 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:04.679053    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:04.850384 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:04.850342    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:04.873228 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:04.873195    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:17:04.873401 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:04.873302    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75b9" podUID="7549f9c6-5988-4771-ae95-99e674252273"
Jul 15 16:17:05.682927 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:05.682892    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:05.846902 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:05.846868    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:05.873331 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:05.873303    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:17:05.873471 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:05.873415    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9k24" podUID="a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd"
Jul 15 16:17:05.873526 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:05.873480    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:17:05.873618 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:05.873593    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t6fc5" podUID="422a9314-d5df-4554-ae6d-2a21856dcb6c"
Jul 15 16:17:06.099421 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:06.099391    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:06.873265 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:06.873078    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:17:06.873869 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:06.873348    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75b9" podUID="7549f9c6-5988-4771-ae95-99e674252273"
Jul 15 16:17:07.013639 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:07.013611    2580 generic.go:356] "Generic (PLEG): container finished" podID="d7494c97-f5f3-47af-a6c7-7f939c53bc39" containerID="8732021f76018fdd267c6b20defd7e287911bfe9456fadb7a897f199f8ea39e2" exitCode=0
Jul 15 16:17:07.013811 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:07.013695    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zrmr" event={"ID":"d7494c97-f5f3-47af-a6c7-7f939c53bc39","Type":"ContainerDied","Data":"8732021f76018fdd267c6b20defd7e287911bfe9456fadb7a897f199f8ea39e2"}
Jul 15 16:17:07.017117 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:07.017100    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-295d8_46356463-bfb1-42b8-8871-71357489a95b/ovn-acl-logging/0.log"
Jul 15 16:17:07.017441 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:07.017423    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-295d8" event={"ID":"46356463-bfb1-42b8-8871-71357489a95b","Type":"ContainerStarted","Data":"fb7c733a8d0739696846b207563dd43f832bd777ae84a3cda9d2015e60001e05"}
Jul 15 16:17:07.017692 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:07.017675    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:17:07.017874 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:07.017859    2580 scope.go:117] "RemoveContainer" containerID="bb926a84c9b5b56dc62cdbe0efec377f607377e29006050a44280b93457e4c22"
Jul 15 16:17:07.033544 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:07.033527    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:17:07.543157 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:07.543125    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:17:07.873060 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:07.873029    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:17:07.873214 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:07.873036    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:17:07.873214 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:07.873132    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t6fc5" podUID="422a9314-d5df-4554-ae6d-2a21856dcb6c"
Jul 15 16:17:07.873284 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:07.873223    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9k24" podUID="a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd"
Jul 15 16:17:08.023107 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:08.023083    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-295d8_46356463-bfb1-42b8-8871-71357489a95b/ovn-acl-logging/0.log"
Jul 15 16:17:08.023455 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:08.023432    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-295d8" event={"ID":"46356463-bfb1-42b8-8871-71357489a95b","Type":"ContainerStarted","Data":"87ba46f832f45a47d7fe00cd9f85b0732691b490af1a55211aa491a1801ab069"}
Jul 15 16:17:08.023622 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:08.023606    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:17:08.038883 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:08.038856    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:17:08.106295 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:08.106238    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-295d8" podStartSLOduration=9.291158667 podStartE2EDuration="29.106217335s" totalImagesPullingTime="19.815058668s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:17:08.06311599 +0000 UTC m=+29.710156527" watchObservedRunningTime="2026-07-15 16:17:08.106217335 +0000 UTC m=+29.753257871"
Jul 15 16:17:08.812441 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:08.812407    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gzwh7"
Jul 15 16:17:08.812644 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:08.812554    2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 15 16:17:08.813071 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:08.813035    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gzwh7"
Jul 15 16:17:08.876062 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:08.876033    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:17:08.876411 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:08.876128    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75b9" podUID="7549f9c6-5988-4771-ae95-99e674252273"
Jul 15 16:17:09.027389 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:09.027351    2580 generic.go:356] "Generic (PLEG): container finished" podID="d7494c97-f5f3-47af-a6c7-7f939c53bc39" containerID="a40aa4a4002a72f85179ed26af1063077ce2b5c726d856f0045c659c826cce47" exitCode=0
Jul 15 16:17:09.027562 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:09.027434    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zrmr" event={"ID":"d7494c97-f5f3-47af-a6c7-7f939c53bc39","Type":"ContainerDied","Data":"a40aa4a4002a72f85179ed26af1063077ce2b5c726d856f0045c659c826cce47"}
Jul 15 16:17:09.873987 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:09.873951    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:17:09.873987 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:09.873975    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:17:09.874181 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:09.874074    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t6fc5" podUID="422a9314-d5df-4554-ae6d-2a21856dcb6c"
Jul 15 16:17:09.874215 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:09.874198    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9k24" podUID="a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd"
Jul 15 16:17:10.873187 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:10.873154    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:17:10.873534 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:10.873262    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75b9" podUID="7549f9c6-5988-4771-ae95-99e674252273"
Jul 15 16:17:11.034204 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:11.034170    2580 generic.go:356] "Generic (PLEG): container finished" podID="d7494c97-f5f3-47af-a6c7-7f939c53bc39" containerID="e2936a5af4ea31968b7d2122faa3dd172fec07406a621986f9f01a449149d795" exitCode=0
Jul 15 16:17:11.034347 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:11.034232    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zrmr" event={"ID":"d7494c97-f5f3-47af-a6c7-7f939c53bc39","Type":"ContainerDied","Data":"e2936a5af4ea31968b7d2122faa3dd172fec07406a621986f9f01a449149d795"}
Jul 15 16:17:11.207394 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:11.207310    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:11.873328 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:11.873297    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:17:11.873786 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:11.873303    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:17:11.873786 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:11.873423    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9k24" podUID="a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd"
Jul 15 16:17:11.873786 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:11.873504    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t6fc5" podUID="422a9314-d5df-4554-ae6d-2a21856dcb6c"
Jul 15 16:17:12.561504 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:12.561270    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-metrics-certs\") pod \"network-metrics-daemon-g9k24\" (UID: \"a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd\") " pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:17:12.561686 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:12.561540    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7549f9c6-5988-4771-ae95-99e674252273-original-pull-secret\") pod \"global-pull-secret-syncer-c75b9\" (UID: \"7549f9c6-5988-4771-ae95-99e674252273\") " pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:17:12.561686 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:12.561479    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:17:12.561686 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:12.561662    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-metrics-certs podName:a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd nodeName:}" failed. No retries permitted until 2026-07-15 16:17:44.561640157 +0000 UTC m=+66.208680689 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-metrics-certs") pod "network-metrics-daemon-g9k24" (UID: "a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 16:17:12.561686 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:12.561672    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 16:17:12.561916 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:12.561721    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7549f9c6-5988-4771-ae95-99e674252273-original-pull-secret podName:7549f9c6-5988-4771-ae95-99e674252273 nodeName:}" failed. No retries permitted until 2026-07-15 16:17:44.561705943 +0000 UTC m=+66.208746480 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7549f9c6-5988-4771-ae95-99e674252273-original-pull-secret") pod "global-pull-secret-syncer-c75b9" (UID: "7549f9c6-5988-4771-ae95-99e674252273") : object "kube-system"/"original-pull-secret" not registered
Jul 15 16:17:12.662772 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:12.662735    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85rfm\" (UniqueName: \"kubernetes.io/projected/422a9314-d5df-4554-ae6d-2a21856dcb6c-kube-api-access-85rfm\") pod \"network-check-target-t6fc5\" (UID: \"422a9314-d5df-4554-ae6d-2a21856dcb6c\") " pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:17:12.662956 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:12.662883    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 16:17:12.662956 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:12.662898    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 16:17:12.662956 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:12.662908    2580 projected.go:196] Error preparing data for projected volume kube-api-access-85rfm for pod openshift-network-diagnostics/network-check-target-t6fc5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:17:12.663119 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:12.662975    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/422a9314-d5df-4554-ae6d-2a21856dcb6c-kube-api-access-85rfm podName:422a9314-d5df-4554-ae6d-2a21856dcb6c nodeName:}" failed. No retries permitted until 2026-07-15 16:17:44.662954293 +0000 UTC m=+66.309994828 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-85rfm" (UniqueName: "kubernetes.io/projected/422a9314-d5df-4554-ae6d-2a21856dcb6c-kube-api-access-85rfm") pod "network-check-target-t6fc5" (UID: "422a9314-d5df-4554-ae6d-2a21856dcb6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 16:17:12.873427 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:12.873354    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:17:12.873779 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:12.873461    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75b9" podUID="7549f9c6-5988-4771-ae95-99e674252273"
Jul 15 16:17:13.873227 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:13.873194    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:17:13.873418 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:13.873236    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:17:13.873418 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:13.873327    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9k24" podUID="a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd"
Jul 15 16:17:13.873829 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:13.873451    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t6fc5" podUID="422a9314-d5df-4554-ae6d-2a21856dcb6c"
Jul 15 16:17:14.876110 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:14.876084    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:17:14.876513 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:14.876193    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75b9" podUID="7549f9c6-5988-4771-ae95-99e674252273"
Jul 15 16:17:15.873865 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:15.873828    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:17:15.874016 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:15.873931    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t6fc5" podUID="422a9314-d5df-4554-ae6d-2a21856dcb6c"
Jul 15 16:17:15.874016 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:15.873983    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:17:15.874105 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:15.874074    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9k24" podUID="a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd"
Jul 15 16:17:16.048588 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:16.048557    2580 generic.go:356] "Generic (PLEG): container finished" podID="d7494c97-f5f3-47af-a6c7-7f939c53bc39" containerID="0c7cb02b047118004a5720042f8e50c2e68d3531435bfb5f73dbd3ca4b632ccd" exitCode=0
Jul 15 16:17:16.048941 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:16.048596    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zrmr" event={"ID":"d7494c97-f5f3-47af-a6c7-7f939c53bc39","Type":"ContainerDied","Data":"0c7cb02b047118004a5720042f8e50c2e68d3531435bfb5f73dbd3ca4b632ccd"}
Jul 15 16:17:16.258695 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:16.258666    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:16.357933 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:16.357900    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:16.872981 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:16.872948    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:17:16.873168 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:16.873048    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75b9" podUID="7549f9c6-5988-4771-ae95-99e674252273"
Jul 15 16:17:17.053230 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:17.053199    2580 generic.go:356] "Generic (PLEG): container finished" podID="d7494c97-f5f3-47af-a6c7-7f939c53bc39" containerID="030aa070988288602a93403919b947a65d6de768a3475a7b7cb102399dde41fc" exitCode=0
Jul 15 16:17:17.053600 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:17.053238    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zrmr" event={"ID":"d7494c97-f5f3-47af-a6c7-7f939c53bc39","Type":"ContainerDied","Data":"030aa070988288602a93403919b947a65d6de768a3475a7b7cb102399dde41fc"}
Jul 15 16:17:17.873488 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:17.873459    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:17:17.873672 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:17.873459    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:17:17.873672 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:17.873586    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9k24" podUID="a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd"
Jul 15 16:17:17.873672 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:17.873619    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t6fc5" podUID="422a9314-d5df-4554-ae6d-2a21856dcb6c"
Jul 15 16:17:18.058892 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:18.058860    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zrmr" event={"ID":"d7494c97-f5f3-47af-a6c7-7f939c53bc39","Type":"ContainerStarted","Data":"eceb773336d0bb5801124a83720d4719178bfb548f931c13241dcc9be98df789"}
Jul 15 16:17:18.874250 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:18.874217    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:17:18.874417 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:18.874297    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75b9" podUID="7549f9c6-5988-4771-ae95-99e674252273"
Jul 15 16:17:19.307368 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:19.307336    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:19.426072 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:19.426040    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:19.575096 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:19.575012    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:19.873613 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:19.873521    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:17:19.873803 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:19.873630    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9k24" podUID="a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd"
Jul 15 16:17:19.873803 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:19.873691    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:17:19.873937 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:19.873838    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t6fc5" podUID="422a9314-d5df-4554-ae6d-2a21856dcb6c"
Jul 15 16:17:20.067313 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:20.067287    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:20.873580 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:20.873551    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:17:20.873978 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:20.873657    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75b9" podUID="7549f9c6-5988-4771-ae95-99e674252273"
Jul 15 16:17:21.873649 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:21.873621    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:17:21.874022 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:21.873734    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9k24" podUID="a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd"
Jul 15 16:17:21.874022 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:21.873789    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:17:21.874022 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:21.873874    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t6fc5" podUID="422a9314-d5df-4554-ae6d-2a21856dcb6c"
Jul 15 16:17:22.292806 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:22.292721    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:22.448220 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:22.448175    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:22.648156 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:22.648123    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:22.749360 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:22.749327    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:22.873825 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:22.873796    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:17:22.874227 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:22.873931    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75b9" podUID="7549f9c6-5988-4771-ae95-99e674252273"
Jul 15 16:17:23.402428 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:23.402364    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6zrmr" podStartSLOduration=14.371392703 podStartE2EDuration="44.402344263s" totalImagesPullingTime="30.03095156s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:16:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:17:18.084729742 +0000 UTC m=+39.731770277" watchObservedRunningTime="2026-07-15 16:17:23.402344263 +0000 UTC m=+45.049384798"
Jul 15 16:17:23.402641 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:23.402544    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-t6fc5"]
Jul 15 16:17:23.402696 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:23.402676    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:17:23.402815 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:23.402788    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t6fc5" podUID="422a9314-d5df-4554-ae6d-2a21856dcb6c"
Jul 15 16:17:23.405134 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:23.405105    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-g9k24"]
Jul 15 16:17:23.405414 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:23.405394    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:17:23.405510 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:23.405485    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9k24" podUID="a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd"
Jul 15 16:17:23.405944 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:23.405912    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c75b9"]
Jul 15 16:17:23.406040 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:23.406014    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:17:23.406134 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:23.406113    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75b9" podUID="7549f9c6-5988-4771-ae95-99e674252273"
Jul 15 16:17:24.873679 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:24.873489    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:17:24.874363 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:24.873489    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:17:24.874363 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:24.873775    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75b9" podUID="7549f9c6-5988-4771-ae95-99e674252273"
Jul 15 16:17:24.874363 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:24.873864    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9k24" podUID="a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd"
Jul 15 16:17:24.874363 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:24.873489    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:17:24.874363 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:24.873949    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t6fc5" podUID="422a9314-d5df-4554-ae6d-2a21856dcb6c"
Jul 15 16:17:26.873476 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:26.873442    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:17:26.873966 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:26.873563    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t6fc5" podUID="422a9314-d5df-4554-ae6d-2a21856dcb6c"
Jul 15 16:17:26.873966 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:26.873632    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:17:26.873966 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:26.873743    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c75b9" podUID="7549f9c6-5988-4771-ae95-99e674252273"
Jul 15 16:17:26.873966 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:26.873788    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:17:26.873966 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:26.873885    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g9k24" podUID="a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd"
Jul 15 16:17:27.697513 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.697485    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-132.ec2.internal" event="NodeReady"
Jul 15 16:17:27.697680 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.697593    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 15 16:17:27.755245 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.755207    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-59798f98df-72z7k"]
Jul 15 16:17:27.778663 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.778635    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59798f98df-72z7k"]
Jul 15 16:17:27.778813 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.778724    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:27.780311 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.780285    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hkgqt"]
Jul 15 16:17:27.781345 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.781319    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 15 16:17:27.781601 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.781583    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 15 16:17:27.781848 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.781832    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-h68pr\""
Jul 15 16:17:27.782451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.782437    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 15 16:17:27.790043 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.790025    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 15 16:17:27.793067 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.793052    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mng27"]
Jul 15 16:17:27.793173 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.793160    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hkgqt"
Jul 15 16:17:27.797087 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.797069    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 15 16:17:27.797355 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.797340    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 15 16:17:27.797465 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.797441    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 15 16:17:27.797465 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.797456    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 15 16:17:27.797627 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.797572    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tw9wx\""
Jul 15 16:17:27.811013 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.810990    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mng27"]
Jul 15 16:17:27.811128 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.811024    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hkgqt"]
Jul 15 16:17:27.811128 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.811096    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mng27"
Jul 15 16:17:27.813544 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.813524    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 15 16:17:27.813666 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.813645    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 15 16:17:27.813748 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.813738    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vkb7c\""
Jul 15 16:17:27.868154 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.868122    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rkbd4"]
Jul 15 16:17:27.881590 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.881561    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7e5e30f2-bd7d-4003-a535-6d2c79e89602-ca-trust-extracted\") pod \"image-registry-59798f98df-72z7k\" (UID: \"7e5e30f2-bd7d-4003-a535-6d2c79e89602\") " pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:27.882086 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.881598    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h879\" (UniqueName: \"kubernetes.io/projected/7a38e9b7-99c1-4867-a5c4-3cb9dead76be-kube-api-access-6h879\") pod \"insights-runtime-extractor-hkgqt\" (UID: \"7a38e9b7-99c1-4867-a5c4-3cb9dead76be\") " pod="openshift-insights/insights-runtime-extractor-hkgqt"
Jul 15 16:17:27.882086 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.881624    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7ce6e188-b85d-426c-bd0a-9390979d9f6b-config-volume\") pod \"dns-default-mng27\" (UID: \"7ce6e188-b85d-426c-bd0a-9390979d9f6b\") " pod="openshift-dns/dns-default-mng27"
Jul 15 16:17:27.882086 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.881707    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7e5e30f2-bd7d-4003-a535-6d2c79e89602-trusted-ca\") pod \"image-registry-59798f98df-72z7k\" (UID: \"7e5e30f2-bd7d-4003-a535-6d2c79e89602\") " pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:27.882086 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.881748    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7e5e30f2-bd7d-4003-a535-6d2c79e89602-bound-sa-token\") pod \"image-registry-59798f98df-72z7k\" (UID: \"7e5e30f2-bd7d-4003-a535-6d2c79e89602\") " pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:27.882086 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.881794    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ce6e188-b85d-426c-bd0a-9390979d9f6b-metrics-tls\") pod \"dns-default-mng27\" (UID: \"7ce6e188-b85d-426c-bd0a-9390979d9f6b\") " pod="openshift-dns/dns-default-mng27"
Jul 15 16:17:27.882086 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.881820    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ce6e188-b85d-426c-bd0a-9390979d9f6b-tmp-dir\") pod \"dns-default-mng27\" (UID: \"7ce6e188-b85d-426c-bd0a-9390979d9f6b\") " pod="openshift-dns/dns-default-mng27"
Jul 15 16:17:27.882086 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.881846    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8kdq\" (UniqueName: \"kubernetes.io/projected/7ce6e188-b85d-426c-bd0a-9390979d9f6b-kube-api-access-k8kdq\") pod \"dns-default-mng27\" (UID: \"7ce6e188-b85d-426c-bd0a-9390979d9f6b\") " pod="openshift-dns/dns-default-mng27"
Jul 15 16:17:27.882086 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.881886    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7a38e9b7-99c1-4867-a5c4-3cb9dead76be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hkgqt\" (UID: \"7a38e9b7-99c1-4867-a5c4-3cb9dead76be\") " pod="openshift-insights/insights-runtime-extractor-hkgqt"
Jul 15 16:17:27.882086 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.881947    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7e5e30f2-bd7d-4003-a535-6d2c79e89602-registry-tls\") pod \"image-registry-59798f98df-72z7k\" (UID: \"7e5e30f2-bd7d-4003-a535-6d2c79e89602\") " pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:27.882086 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.881972    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7a38e9b7-99c1-4867-a5c4-3cb9dead76be-crio-socket\") pod \"insights-runtime-extractor-hkgqt\" (UID: \"7a38e9b7-99c1-4867-a5c4-3cb9dead76be\") " pod="openshift-insights/insights-runtime-extractor-hkgqt"
Jul 15 16:17:27.882086 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.881995    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a38e9b7-99c1-4867-a5c4-3cb9dead76be-data-volume\") pod \"insights-runtime-extractor-hkgqt\" (UID: \"7a38e9b7-99c1-4867-a5c4-3cb9dead76be\") " pod="openshift-insights/insights-runtime-extractor-hkgqt"
Jul 15 16:17:27.882086 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.882033    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7e5e30f2-bd7d-4003-a535-6d2c79e89602-registry-certificates\") pod \"image-registry-59798f98df-72z7k\" (UID: \"7e5e30f2-bd7d-4003-a535-6d2c79e89602\") " pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:27.882086 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.882059    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7e5e30f2-bd7d-4003-a535-6d2c79e89602-installation-pull-secrets\") pod \"image-registry-59798f98df-72z7k\" (UID: \"7e5e30f2-bd7d-4003-a535-6d2c79e89602\") " pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:27.882086 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.882081    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z282h\" (UniqueName: \"kubernetes.io/projected/7e5e30f2-bd7d-4003-a535-6d2c79e89602-kube-api-access-z282h\") pod \"image-registry-59798f98df-72z7k\" (UID: \"7e5e30f2-bd7d-4003-a535-6d2c79e89602\") " pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:27.882538 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.882118    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7a38e9b7-99c1-4867-a5c4-3cb9dead76be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hkgqt\" (UID: \"7a38e9b7-99c1-4867-a5c4-3cb9dead76be\") " pod="openshift-insights/insights-runtime-extractor-hkgqt"
Jul 15 16:17:27.882538 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.882227    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7e5e30f2-bd7d-4003-a535-6d2c79e89602-image-registry-private-configuration\") pod \"image-registry-59798f98df-72z7k\" (UID: \"7e5e30f2-bd7d-4003-a535-6d2c79e89602\") " pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:27.887364 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.887343    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rkbd4"]
Jul 15 16:17:27.887456 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.887439    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rkbd4"
Jul 15 16:17:27.892249 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.892228    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7vwjh\""
Jul 15 16:17:27.892490 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.892473    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 15 16:17:27.892578 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.892547    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 15 16:17:27.892709 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.892694    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 15 16:17:27.956229 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.956145    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:27.983004 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.982969    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7e5e30f2-bd7d-4003-a535-6d2c79e89602-registry-tls\") pod \"image-registry-59798f98df-72z7k\" (UID: \"7e5e30f2-bd7d-4003-a535-6d2c79e89602\") " pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:27.983184 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.983008    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7a38e9b7-99c1-4867-a5c4-3cb9dead76be-crio-socket\") pod \"insights-runtime-extractor-hkgqt\" (UID: \"7a38e9b7-99c1-4867-a5c4-3cb9dead76be\") " pod="openshift-insights/insights-runtime-extractor-hkgqt"
Jul 15 16:17:27.983184 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.983035    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a38e9b7-99c1-4867-a5c4-3cb9dead76be-data-volume\") pod \"insights-runtime-extractor-hkgqt\" (UID: \"7a38e9b7-99c1-4867-a5c4-3cb9dead76be\") " pod="openshift-insights/insights-runtime-extractor-hkgqt"
Jul 15 16:17:27.983304 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.983171    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7e5e30f2-bd7d-4003-a535-6d2c79e89602-registry-certificates\") pod \"image-registry-59798f98df-72z7k\" (UID: \"7e5e30f2-bd7d-4003-a535-6d2c79e89602\") " pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:27.983304 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.983212    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7a38e9b7-99c1-4867-a5c4-3cb9dead76be-crio-socket\") pod \"insights-runtime-extractor-hkgqt\" (UID: \"7a38e9b7-99c1-4867-a5c4-3cb9dead76be\") " pod="openshift-insights/insights-runtime-extractor-hkgqt"
Jul 15 16:17:27.983304 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.983212    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7e5e30f2-bd7d-4003-a535-6d2c79e89602-installation-pull-secrets\") pod \"image-registry-59798f98df-72z7k\" (UID: \"7e5e30f2-bd7d-4003-a535-6d2c79e89602\") " pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:27.983304 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.983271    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z282h\" (UniqueName: \"kubernetes.io/projected/7e5e30f2-bd7d-4003-a535-6d2c79e89602-kube-api-access-z282h\") pod \"image-registry-59798f98df-72z7k\" (UID: \"7e5e30f2-bd7d-4003-a535-6d2c79e89602\") " pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:27.983506 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.983313    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7a38e9b7-99c1-4867-a5c4-3cb9dead76be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hkgqt\" (UID: \"7a38e9b7-99c1-4867-a5c4-3cb9dead76be\") " pod="openshift-insights/insights-runtime-extractor-hkgqt"
Jul 15 16:17:27.983506 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.983339    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d99hb\" (UniqueName: \"kubernetes.io/projected/ce92f3bb-14db-4828-af08-055c33a25a1d-kube-api-access-d99hb\") pod \"ingress-canary-rkbd4\" (UID: \"ce92f3bb-14db-4828-af08-055c33a25a1d\") " pod="openshift-ingress-canary/ingress-canary-rkbd4"
Jul 15 16:17:27.983506 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.983357    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a38e9b7-99c1-4867-a5c4-3cb9dead76be-data-volume\") pod \"insights-runtime-extractor-hkgqt\" (UID: \"7a38e9b7-99c1-4867-a5c4-3cb9dead76be\") " pod="openshift-insights/insights-runtime-extractor-hkgqt"
Jul 15 16:17:27.983506 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.983403    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7e5e30f2-bd7d-4003-a535-6d2c79e89602-image-registry-private-configuration\") pod \"image-registry-59798f98df-72z7k\" (UID: \"7e5e30f2-bd7d-4003-a535-6d2c79e89602\") " pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:27.983506 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.983452    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7e5e30f2-bd7d-4003-a535-6d2c79e89602-ca-trust-extracted\") pod \"image-registry-59798f98df-72z7k\" (UID: \"7e5e30f2-bd7d-4003-a535-6d2c79e89602\") " pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:27.983506 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.983480    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h879\" (UniqueName: \"kubernetes.io/projected/7a38e9b7-99c1-4867-a5c4-3cb9dead76be-kube-api-access-6h879\") pod \"insights-runtime-extractor-hkgqt\" (UID: \"7a38e9b7-99c1-4867-a5c4-3cb9dead76be\") " pod="openshift-insights/insights-runtime-extractor-hkgqt"
Jul 15 16:17:27.983506 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.983505    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7ce6e188-b85d-426c-bd0a-9390979d9f6b-config-volume\") pod \"dns-default-mng27\" (UID: \"7ce6e188-b85d-426c-bd0a-9390979d9f6b\") " pod="openshift-dns/dns-default-mng27"
Jul 15 16:17:27.983894 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.983536    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7e5e30f2-bd7d-4003-a535-6d2c79e89602-trusted-ca\") pod \"image-registry-59798f98df-72z7k\" (UID: \"7e5e30f2-bd7d-4003-a535-6d2c79e89602\") " pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:27.983894 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.983559    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7e5e30f2-bd7d-4003-a535-6d2c79e89602-bound-sa-token\") pod \"image-registry-59798f98df-72z7k\" (UID: \"7e5e30f2-bd7d-4003-a535-6d2c79e89602\") " pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:27.983894 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.983584    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ce6e188-b85d-426c-bd0a-9390979d9f6b-metrics-tls\") pod \"dns-default-mng27\" (UID: \"7ce6e188-b85d-426c-bd0a-9390979d9f6b\") " pod="openshift-dns/dns-default-mng27"
Jul 15 16:17:27.983894 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.983613    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ce6e188-b85d-426c-bd0a-9390979d9f6b-tmp-dir\") pod \"dns-default-mng27\" (UID: \"7ce6e188-b85d-426c-bd0a-9390979d9f6b\") " pod="openshift-dns/dns-default-mng27"
Jul 15 16:17:27.983894 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.983637    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8kdq\" (UniqueName: \"kubernetes.io/projected/7ce6e188-b85d-426c-bd0a-9390979d9f6b-kube-api-access-k8kdq\") pod \"dns-default-mng27\" (UID: \"7ce6e188-b85d-426c-bd0a-9390979d9f6b\") " pod="openshift-dns/dns-default-mng27"
Jul 15 16:17:27.983894 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.983672    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce92f3bb-14db-4828-af08-055c33a25a1d-cert\") pod \"ingress-canary-rkbd4\" (UID: \"ce92f3bb-14db-4828-af08-055c33a25a1d\") " pod="openshift-ingress-canary/ingress-canary-rkbd4"
Jul 15 16:17:27.983894 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.983703    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7a38e9b7-99c1-4867-a5c4-3cb9dead76be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hkgqt\" (UID: \"7a38e9b7-99c1-4867-a5c4-3cb9dead76be\") " pod="openshift-insights/insights-runtime-extractor-hkgqt"
Jul 15 16:17:27.984237 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.983987    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7a38e9b7-99c1-4867-a5c4-3cb9dead76be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hkgqt\" (UID: \"7a38e9b7-99c1-4867-a5c4-3cb9dead76be\") " pod="openshift-insights/insights-runtime-extractor-hkgqt"
Jul 15 16:17:27.984237 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.984109    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7e5e30f2-bd7d-4003-a535-6d2c79e89602-ca-trust-extracted\") pod \"image-registry-59798f98df-72z7k\" (UID: \"7e5e30f2-bd7d-4003-a535-6d2c79e89602\") " pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:27.984557 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.984510    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7e5e30f2-bd7d-4003-a535-6d2c79e89602-registry-certificates\") pod \"image-registry-59798f98df-72z7k\" (UID: \"7e5e30f2-bd7d-4003-a535-6d2c79e89602\") " pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:27.984977 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.984952    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7e5e30f2-bd7d-4003-a535-6d2c79e89602-trusted-ca\") pod \"image-registry-59798f98df-72z7k\" (UID: \"7e5e30f2-bd7d-4003-a535-6d2c79e89602\") " pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:27.985111 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.985092    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ce6e188-b85d-426c-bd0a-9390979d9f6b-tmp-dir\") pod \"dns-default-mng27\" (UID: \"7ce6e188-b85d-426c-bd0a-9390979d9f6b\") " pod="openshift-dns/dns-default-mng27"
Jul 15 16:17:27.985414 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.985390    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7ce6e188-b85d-426c-bd0a-9390979d9f6b-config-volume\") pod \"dns-default-mng27\" (UID: \"7ce6e188-b85d-426c-bd0a-9390979d9f6b\") " pod="openshift-dns/dns-default-mng27"
Jul 15 16:17:27.987557 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.987537    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7a38e9b7-99c1-4867-a5c4-3cb9dead76be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hkgqt\" (UID: \"7a38e9b7-99c1-4867-a5c4-3cb9dead76be\") " pod="openshift-insights/insights-runtime-extractor-hkgqt"
Jul 15 16:17:27.987660 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.987540    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7e5e30f2-bd7d-4003-a535-6d2c79e89602-installation-pull-secrets\") pod \"image-registry-59798f98df-72z7k\" (UID: \"7e5e30f2-bd7d-4003-a535-6d2c79e89602\") " pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:27.987660 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.987577    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7e5e30f2-bd7d-4003-a535-6d2c79e89602-image-registry-private-configuration\") pod \"image-registry-59798f98df-72z7k\" (UID: \"7e5e30f2-bd7d-4003-a535-6d2c79e89602\") " pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:27.987660 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.987584    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ce6e188-b85d-426c-bd0a-9390979d9f6b-metrics-tls\") pod \"dns-default-mng27\" (UID: \"7ce6e188-b85d-426c-bd0a-9390979d9f6b\") " pod="openshift-dns/dns-default-mng27"
Jul 15 16:17:27.987660 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.987652    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7e5e30f2-bd7d-4003-a535-6d2c79e89602-registry-tls\") pod \"image-registry-59798f98df-72z7k\" (UID: \"7e5e30f2-bd7d-4003-a535-6d2c79e89602\") " pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:27.993400 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.993380    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h879\" (UniqueName: \"kubernetes.io/projected/7a38e9b7-99c1-4867-a5c4-3cb9dead76be-kube-api-access-6h879\") pod \"insights-runtime-extractor-hkgqt\" (UID: \"7a38e9b7-99c1-4867-a5c4-3cb9dead76be\") " pod="openshift-insights/insights-runtime-extractor-hkgqt"
Jul 15 16:17:27.993869 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.993829    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7e5e30f2-bd7d-4003-a535-6d2c79e89602-bound-sa-token\") pod \"image-registry-59798f98df-72z7k\" (UID: \"7e5e30f2-bd7d-4003-a535-6d2c79e89602\") " pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:27.994026 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.994002    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8kdq\" (UniqueName: \"kubernetes.io/projected/7ce6e188-b85d-426c-bd0a-9390979d9f6b-kube-api-access-k8kdq\") pod \"dns-default-mng27\" (UID: \"7ce6e188-b85d-426c-bd0a-9390979d9f6b\") " pod="openshift-dns/dns-default-mng27"
Jul 15 16:17:27.994297 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:27.994275    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z282h\" (UniqueName: \"kubernetes.io/projected/7e5e30f2-bd7d-4003-a535-6d2c79e89602-kube-api-access-z282h\") pod \"image-registry-59798f98df-72z7k\" (UID: \"7e5e30f2-bd7d-4003-a535-6d2c79e89602\") " pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:28.084214 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:28.084180    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d99hb\" (UniqueName: \"kubernetes.io/projected/ce92f3bb-14db-4828-af08-055c33a25a1d-kube-api-access-d99hb\") pod \"ingress-canary-rkbd4\" (UID: \"ce92f3bb-14db-4828-af08-055c33a25a1d\") " pod="openshift-ingress-canary/ingress-canary-rkbd4"
Jul 15 16:17:28.084360 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:28.084234    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce92f3bb-14db-4828-af08-055c33a25a1d-cert\") pod \"ingress-canary-rkbd4\" (UID: \"ce92f3bb-14db-4828-af08-055c33a25a1d\") " pod="openshift-ingress-canary/ingress-canary-rkbd4"
Jul 15 16:17:28.089659 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:28.089588    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:28.096359 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:28.096329    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce92f3bb-14db-4828-af08-055c33a25a1d-cert\") pod \"ingress-canary-rkbd4\" (UID: \"ce92f3bb-14db-4828-af08-055c33a25a1d\") " pod="openshift-ingress-canary/ingress-canary-rkbd4"
Jul 15 16:17:28.099158 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:28.099033    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d99hb\" (UniqueName: \"kubernetes.io/projected/ce92f3bb-14db-4828-af08-055c33a25a1d-kube-api-access-d99hb\") pod \"ingress-canary-rkbd4\" (UID: \"ce92f3bb-14db-4828-af08-055c33a25a1d\") " pod="openshift-ingress-canary/ingress-canary-rkbd4"
Jul 15 16:17:28.101691 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:28.101673    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hkgqt"
Jul 15 16:17:28.118892 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:28.118867    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mng27"
Jul 15 16:17:28.196330 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:28.196137    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rkbd4"
Jul 15 16:17:28.290399 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:28.289460    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:28.313187 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:28.313151    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hkgqt"]
Jul 15 16:17:28.322547 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:28.322474    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59798f98df-72z7k"]
Jul 15 16:17:28.326193 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:28.326172    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mng27"]
Jul 15 16:17:28.328872 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:17:28.328844    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e5e30f2_bd7d_4003_a535_6d2c79e89602.slice/crio-6b2c6565e5f022de68adeb70c8d72a8ef84a22cb2ba1a31cee1730e8024deb04 WatchSource:0}: Error finding container 6b2c6565e5f022de68adeb70c8d72a8ef84a22cb2ba1a31cee1730e8024deb04: Status 404 returned error can't find the container with id 6b2c6565e5f022de68adeb70c8d72a8ef84a22cb2ba1a31cee1730e8024deb04
Jul 15 16:17:28.329129 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:17:28.329109    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ce6e188_b85d_426c_bd0a_9390979d9f6b.slice/crio-51f6d6aaf3510d31da94ff455902832a50ca41c7939dce825f1867b67c7cb928 WatchSource:0}: Error finding container 51f6d6aaf3510d31da94ff455902832a50ca41c7939dce825f1867b67c7cb928: Status 404 returned error can't find the container with id 51f6d6aaf3510d31da94ff455902832a50ca41c7939dce825f1867b67c7cb928
Jul 15 16:17:28.365476 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:28.365452    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rkbd4"]
Jul 15 16:17:28.371341 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:28.371319    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:28.376120 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:17:28.376096    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce92f3bb_14db_4828_af08_055c33a25a1d.slice/crio-8fa5ffffb751ca5dd00c16ea8b58380bead7179275a42b0554fc28f538fb63f8 WatchSource:0}: Error finding container 8fa5ffffb751ca5dd00c16ea8b58380bead7179275a42b0554fc28f538fb63f8: Status 404 returned error can't find the container with id 8fa5ffffb751ca5dd00c16ea8b58380bead7179275a42b0554fc28f538fb63f8
Jul 15 16:17:28.879332 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:28.879295    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:17:28.879744 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:28.879722    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:17:28.879744 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:28.879770    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:17:28.884160 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:28.884135    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 16:17:28.884837 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:28.884372    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wsr6k\""
Jul 15 16:17:28.884837 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:28.884389    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 16:17:28.884837 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:28.884549    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 16:17:28.884837 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:28.884618    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 16:17:28.884837 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:28.884722    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dmv8w\""
Jul 15 16:17:29.087051 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:29.087015    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hkgqt" event={"ID":"7a38e9b7-99c1-4867-a5c4-3cb9dead76be","Type":"ContainerStarted","Data":"9c8a110891aefe1fac56dca8b1c693f982a70546831318090e14a7058ea51ab4"}
Jul 15 16:17:29.089053 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:29.089023    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mng27" event={"ID":"7ce6e188-b85d-426c-bd0a-9390979d9f6b","Type":"ContainerStarted","Data":"51f6d6aaf3510d31da94ff455902832a50ca41c7939dce825f1867b67c7cb928"}
Jul 15 16:17:29.091159 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:29.091131    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59798f98df-72z7k" event={"ID":"7e5e30f2-bd7d-4003-a535-6d2c79e89602","Type":"ContainerStarted","Data":"f98bfa7eafbb3b45894a6ebbc824b2cbb5d25e9b4ac454b78a48e0fa101c8c90"}
Jul 15 16:17:29.091272 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:29.091179    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59798f98df-72z7k" event={"ID":"7e5e30f2-bd7d-4003-a535-6d2c79e89602","Type":"ContainerStarted","Data":"6b2c6565e5f022de68adeb70c8d72a8ef84a22cb2ba1a31cee1730e8024deb04"}
Jul 15 16:17:29.091965 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:29.091941    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:29.094734 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:29.094713    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rkbd4" event={"ID":"ce92f3bb-14db-4828-af08-055c33a25a1d","Type":"ContainerStarted","Data":"8fa5ffffb751ca5dd00c16ea8b58380bead7179275a42b0554fc28f538fb63f8"}
Jul 15 16:17:29.118407 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:29.118346    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-59798f98df-72z7k" podStartSLOduration=7.118324556 podStartE2EDuration="7.118324556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:17:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:17:29.117540379 +0000 UTC m=+50.764580976" watchObservedRunningTime="2026-07-15 16:17:29.118324556 +0000 UTC m=+50.765365094"
Jul 15 16:17:30.621337 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:30.621262    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:30.873473 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:30.873389    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:30.974657 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:30.974620    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:31.720616 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:31.720584    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:31.849014 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:31.848979    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:34.287006 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:34.286958    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:34.359100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:34.359014    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:35.114537 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:35.114305    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mng27" event={"ID":"7ce6e188-b85d-426c-bd0a-9390979d9f6b","Type":"ContainerStarted","Data":"2bc60389e80c66025b011237003095f8f04c7f8e700662046a2a4f69badcc8ad"}
Jul 15 16:17:35.114735 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:35.114566    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mng27" event={"ID":"7ce6e188-b85d-426c-bd0a-9390979d9f6b","Type":"ContainerStarted","Data":"2310c8d990bea75bf674388c5eb82a84acf59658aad91ed5336177f47a9926cd"}
Jul 15 16:17:35.114735 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:35.114600    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mng27"
Jul 15 16:17:35.115803 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:35.115778    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rkbd4" event={"ID":"ce92f3bb-14db-4828-af08-055c33a25a1d","Type":"ContainerStarted","Data":"8d8d7ef67bd87a6bb5a39bf951b86c8a15a7e5e18e02446c54b9abbf5a5e991f"}
Jul 15 16:17:35.117127 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:35.117108    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hkgqt" event={"ID":"7a38e9b7-99c1-4867-a5c4-3cb9dead76be","Type":"ContainerStarted","Data":"c00c84946679705ea8349f55dfb486d0a3374e845f05337a2996b43559eb6180"}
Jul 15 16:17:35.137710 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:35.137665    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mng27" podStartSLOduration=1.705267721 podStartE2EDuration="8.137650836s" totalImagesPullingTime="6.432383115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:17:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:17:35.13731268 +0000 UTC m=+56.784353262" watchObservedRunningTime="2026-07-15 16:17:35.137650836 +0000 UTC m=+56.784691371"
Jul 15 16:17:35.381667 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:35.381590    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:36.882208 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:36.882151    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:39.133772 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:39.133733    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hkgqt" event={"ID":"7a38e9b7-99c1-4867-a5c4-3cb9dead76be","Type":"ContainerStarted","Data":"a64517a1ef8b30aca8ea8c3b03598f506a9d3dff37f7d228fe38b122eab75d13"}
Jul 15 16:17:39.797063 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:39.797027    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:40.052670 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.052639    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-295d8"
Jul 15 16:17:40.111131 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.111064    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rkbd4" podStartSLOduration=6.7244008730000004 podStartE2EDuration="13.111047576s" totalImagesPullingTime="6.386646703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:17:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:17:35.162691604 +0000 UTC m=+56.809732140" watchObservedRunningTime="2026-07-15 16:17:40.111047576 +0000 UTC m=+61.758088112"
Jul 15 16:17:40.113545 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.113517    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:40.116162 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.116140    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-2sqh9"]
Jul 15 16:17:40.137337 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.137309    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-2sqh9"]
Jul 15 16:17:40.137740 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.137413    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-2sqh9"
Jul 15 16:17:40.140916 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.140895    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 15 16:17:40.141308 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.141289    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-n6rvf\""
Jul 15 16:17:40.143099 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.143078    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 15 16:17:40.185208 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.185175    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knmhp\" (UniqueName: \"kubernetes.io/projected/44adf4d8-cfe8-4db9-8c4d-ac493a58493b-kube-api-access-knmhp\") pod \"downloads-68c9cbb497-2sqh9\" (UID: \"44adf4d8-cfe8-4db9-8c4d-ac493a58493b\") " pod="openshift-console/downloads-68c9cbb497-2sqh9"
Jul 15 16:17:40.274015 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.273958    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:40.285820 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.285788    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knmhp\" (UniqueName: \"kubernetes.io/projected/44adf4d8-cfe8-4db9-8c4d-ac493a58493b-kube-api-access-knmhp\") pod \"downloads-68c9cbb497-2sqh9\" (UID: \"44adf4d8-cfe8-4db9-8c4d-ac493a58493b\") " pod="openshift-console/downloads-68c9cbb497-2sqh9"
Jul 15 16:17:40.295860 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.295827    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-2z6kg"]
Jul 15 16:17:40.315949 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.315876    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-2z6kg"
Jul 15 16:17:40.316707 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.316684    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knmhp\" (UniqueName: \"kubernetes.io/projected/44adf4d8-cfe8-4db9-8c4d-ac493a58493b-kube-api-access-knmhp\") pod \"downloads-68c9cbb497-2sqh9\" (UID: \"44adf4d8-cfe8-4db9-8c4d-ac493a58493b\") " pod="openshift-console/downloads-68c9cbb497-2sqh9"
Jul 15 16:17:40.323608 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.323579    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 15 16:17:40.323691 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.323632    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 15 16:17:40.323691 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.323632    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-szxpp\""
Jul 15 16:17:40.323963 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.323950    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 15 16:17:40.324292 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.324277    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 15 16:17:40.324505 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.324492    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 15 16:17:40.324546 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.324537    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 15 16:17:40.338772 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.338739    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-2z6kg"]
Jul 15 16:17:40.339359 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.339343    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7g2q8"]
Jul 15 16:17:40.353674 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.353652    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.358629 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.358610    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 15 16:17:40.358723 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.358708    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6pvz6\""
Jul 15 16:17:40.359269 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.359104    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 15 16:17:40.361869 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.361852    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 15 16:17:40.386554 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.386527    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2241b2de-66eb-497c-b71c-34b725a04409-root\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.386554 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.386552    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2241b2de-66eb-497c-b71c-34b725a04409-node-exporter-accelerators-collector-config\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.386787 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.386572    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ab4d8ae-d58c-4d06-b208-73a036fd4f69-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-2z6kg\" (UID: \"7ab4d8ae-d58c-4d06-b208-73a036fd4f69\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2z6kg"
Jul 15 16:17:40.386787 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.386608    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2241b2de-66eb-497c-b71c-34b725a04409-node-exporter-textfile\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.386787 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.386683    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2241b2de-66eb-497c-b71c-34b725a04409-sys\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.386787 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.386734    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7ab4d8ae-d58c-4d06-b208-73a036fd4f69-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-2z6kg\" (UID: \"7ab4d8ae-d58c-4d06-b208-73a036fd4f69\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2z6kg"
Jul 15 16:17:40.386970 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.386791    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ab4d8ae-d58c-4d06-b208-73a036fd4f69-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-2z6kg\" (UID: \"7ab4d8ae-d58c-4d06-b208-73a036fd4f69\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2z6kg"
Jul 15 16:17:40.386970 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.386821    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2241b2de-66eb-497c-b71c-34b725a04409-metrics-client-ca\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.386970 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.386845    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ab4d8ae-d58c-4d06-b208-73a036fd4f69-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-2z6kg\" (UID: \"7ab4d8ae-d58c-4d06-b208-73a036fd4f69\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2z6kg"
Jul 15 16:17:40.386970 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.386897    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2241b2de-66eb-497c-b71c-34b725a04409-node-exporter-tls\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.386970 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.386944    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2241b2de-66eb-497c-b71c-34b725a04409-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.387159 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.387004    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2241b2de-66eb-497c-b71c-34b725a04409-node-exporter-wtmp\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.387159 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.387042    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-978n6\" (UniqueName: \"kubernetes.io/projected/7ab4d8ae-d58c-4d06-b208-73a036fd4f69-kube-api-access-978n6\") pod \"kube-state-metrics-785778ff74-2z6kg\" (UID: \"7ab4d8ae-d58c-4d06-b208-73a036fd4f69\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2z6kg"
Jul 15 16:17:40.387159 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.387077    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwjcz\" (UniqueName: \"kubernetes.io/projected/2241b2de-66eb-497c-b71c-34b725a04409-kube-api-access-gwjcz\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.387159 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.387103    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7ab4d8ae-d58c-4d06-b208-73a036fd4f69-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-2z6kg\" (UID: \"7ab4d8ae-d58c-4d06-b208-73a036fd4f69\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2z6kg"
Jul 15 16:17:40.446672 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.446641    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:40.448476 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.448451    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-2sqh9"
Jul 15 16:17:40.488446 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.488421    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwjcz\" (UniqueName: \"kubernetes.io/projected/2241b2de-66eb-497c-b71c-34b725a04409-kube-api-access-gwjcz\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.488591 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.488464    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7ab4d8ae-d58c-4d06-b208-73a036fd4f69-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-2z6kg\" (UID: \"7ab4d8ae-d58c-4d06-b208-73a036fd4f69\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2z6kg"
Jul 15 16:17:40.488591 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.488514    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2241b2de-66eb-497c-b71c-34b725a04409-root\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.488591 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.488553    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2241b2de-66eb-497c-b71c-34b725a04409-node-exporter-accelerators-collector-config\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.488743 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.488593    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ab4d8ae-d58c-4d06-b208-73a036fd4f69-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-2z6kg\" (UID: \"7ab4d8ae-d58c-4d06-b208-73a036fd4f69\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2z6kg"
Jul 15 16:17:40.488743 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.488656    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2241b2de-66eb-497c-b71c-34b725a04409-root\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.488743 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.488727    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2241b2de-66eb-497c-b71c-34b725a04409-node-exporter-textfile\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.488906 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.488789    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2241b2de-66eb-497c-b71c-34b725a04409-sys\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.488906 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.488830    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7ab4d8ae-d58c-4d06-b208-73a036fd4f69-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-2z6kg\" (UID: \"7ab4d8ae-d58c-4d06-b208-73a036fd4f69\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2z6kg"
Jul 15 16:17:40.488906 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.488855    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ab4d8ae-d58c-4d06-b208-73a036fd4f69-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-2z6kg\" (UID: \"7ab4d8ae-d58c-4d06-b208-73a036fd4f69\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2z6kg"
Jul 15 16:17:40.488906 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.488888    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2241b2de-66eb-497c-b71c-34b725a04409-metrics-client-ca\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.489101 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.488916    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ab4d8ae-d58c-4d06-b208-73a036fd4f69-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-2z6kg\" (UID: \"7ab4d8ae-d58c-4d06-b208-73a036fd4f69\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2z6kg"
Jul 15 16:17:40.489101 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.488942    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2241b2de-66eb-497c-b71c-34b725a04409-node-exporter-tls\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.489101 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.488975    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2241b2de-66eb-497c-b71c-34b725a04409-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.489101 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.489057    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2241b2de-66eb-497c-b71c-34b725a04409-node-exporter-wtmp\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.489292 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.489129    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-978n6\" (UniqueName: \"kubernetes.io/projected/7ab4d8ae-d58c-4d06-b208-73a036fd4f69-kube-api-access-978n6\") pod \"kube-state-metrics-785778ff74-2z6kg\" (UID: \"7ab4d8ae-d58c-4d06-b208-73a036fd4f69\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2z6kg"
Jul 15 16:17:40.489443 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.489415    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2241b2de-66eb-497c-b71c-34b725a04409-sys\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.490063 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.489533    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2241b2de-66eb-497c-b71c-34b725a04409-node-exporter-accelerators-collector-config\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.490063 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.489583    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7ab4d8ae-d58c-4d06-b208-73a036fd4f69-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-2z6kg\" (UID: \"7ab4d8ae-d58c-4d06-b208-73a036fd4f69\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2z6kg"
Jul 15 16:17:40.490063 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.489678    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2241b2de-66eb-497c-b71c-34b725a04409-node-exporter-textfile\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.490063 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.489739    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2241b2de-66eb-497c-b71c-34b725a04409-metrics-client-ca\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.490063 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:40.489815    2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 15 16:17:40.490063 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:40.489870    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2241b2de-66eb-497c-b71c-34b725a04409-node-exporter-tls podName:2241b2de-66eb-497c-b71c-34b725a04409 nodeName:}" failed. No retries permitted until 2026-07-15 16:17:40.989850768 +0000 UTC m=+62.636891298 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/2241b2de-66eb-497c-b71c-34b725a04409-node-exporter-tls") pod "node-exporter-7g2q8" (UID: "2241b2de-66eb-497c-b71c-34b725a04409") : secret "node-exporter-tls" not found
Jul 15 16:17:40.490063 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.489997    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7ab4d8ae-d58c-4d06-b208-73a036fd4f69-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-2z6kg\" (UID: \"7ab4d8ae-d58c-4d06-b208-73a036fd4f69\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2z6kg"
Jul 15 16:17:40.490417 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.490113    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ab4d8ae-d58c-4d06-b208-73a036fd4f69-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-2z6kg\" (UID: \"7ab4d8ae-d58c-4d06-b208-73a036fd4f69\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2z6kg"
Jul 15 16:17:40.490417 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.490144    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2241b2de-66eb-497c-b71c-34b725a04409-node-exporter-wtmp\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.492448 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.492424    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ab4d8ae-d58c-4d06-b208-73a036fd4f69-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-2z6kg\" (UID: \"7ab4d8ae-d58c-4d06-b208-73a036fd4f69\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2z6kg"
Jul 15 16:17:40.492549 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.492458    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2241b2de-66eb-497c-b71c-34b725a04409-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.492717 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.492685    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ab4d8ae-d58c-4d06-b208-73a036fd4f69-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-2z6kg\" (UID: \"7ab4d8ae-d58c-4d06-b208-73a036fd4f69\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2z6kg"
Jul 15 16:17:40.504496 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.504476    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwjcz\" (UniqueName: \"kubernetes.io/projected/2241b2de-66eb-497c-b71c-34b725a04409-kube-api-access-gwjcz\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.505287 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.505258    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-978n6\" (UniqueName: \"kubernetes.io/projected/7ab4d8ae-d58c-4d06-b208-73a036fd4f69-kube-api-access-978n6\") pod \"kube-state-metrics-785778ff74-2z6kg\" (UID: \"7ab4d8ae-d58c-4d06-b208-73a036fd4f69\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2z6kg"
Jul 15 16:17:40.559121 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.559082    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:40.629315 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.629222    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-2z6kg"
Jul 15 16:17:40.700325 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.700287    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:40.980016 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.979943    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:40.995951 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:40.995914    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2241b2de-66eb-497c-b71c-34b725a04409-node-exporter-tls\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:40.996145 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:40.996118    2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 15 16:17:40.996276 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:17:40.996205    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2241b2de-66eb-497c-b71c-34b725a04409-node-exporter-tls podName:2241b2de-66eb-497c-b71c-34b725a04409 nodeName:}" failed. No retries permitted until 2026-07-15 16:17:41.996182116 +0000 UTC m=+63.643222630 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/2241b2de-66eb-497c-b71c-34b725a04409-node-exporter-tls") pod "node-exporter-7g2q8" (UID: "2241b2de-66eb-497c-b71c-34b725a04409") : secret "node-exporter-tls" not found
Jul 15 16:17:41.079333 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:41.079296    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:41.182251 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:41.182219    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:41.306912 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:41.306877    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:41.424518 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:41.424483    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:42.004210 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:42.004176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2241b2de-66eb-497c-b71c-34b725a04409-node-exporter-tls\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:42.006492 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:42.006459    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2241b2de-66eb-497c-b71c-34b725a04409-node-exporter-tls\") pod \"node-exporter-7g2q8\" (UID: \"2241b2de-66eb-497c-b71c-34b725a04409\") " pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:42.162501 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:42.162477    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7g2q8"
Jul 15 16:17:42.171477 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:17:42.171453    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2241b2de_66eb_497c_b71c_34b725a04409.slice/crio-af26e6d8f2175d77c2abb626a1eaf9d6425456c4adb172212d2f63df48d64f96 WatchSource:0}: Error finding container af26e6d8f2175d77c2abb626a1eaf9d6425456c4adb172212d2f63df48d64f96: Status 404 returned error can't find the container with id af26e6d8f2175d77c2abb626a1eaf9d6425456c4adb172212d2f63df48d64f96
Jul 15 16:17:42.179409 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:42.179386    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-2z6kg"]
Jul 15 16:17:42.182274 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:17:42.182250    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ab4d8ae_d58c_4d06_b208_73a036fd4f69.slice/crio-da148f0c605a96fb8e54062fbde7b6e551ccddc880c17f245c0748746e9f3e32 WatchSource:0}: Error finding container da148f0c605a96fb8e54062fbde7b6e551ccddc880c17f245c0748746e9f3e32: Status 404 returned error can't find the container with id da148f0c605a96fb8e54062fbde7b6e551ccddc880c17f245c0748746e9f3e32
Jul 15 16:17:42.186805 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:42.186747    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-2sqh9"]
Jul 15 16:17:42.189109 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:17:42.189085    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44adf4d8_cfe8_4db9_8c4d_ac493a58493b.slice/crio-2a27edf3d056932fc5ed188eff6ebc0e37c560a5814b6a69dbca29154b7dcc79 WatchSource:0}: Error finding container 2a27edf3d056932fc5ed188eff6ebc0e37c560a5814b6a69dbca29154b7dcc79: Status 404 returned error can't find the container with id 2a27edf3d056932fc5ed188eff6ebc0e37c560a5814b6a69dbca29154b7dcc79
Jul 15 16:17:42.203338 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:42.203310    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:42.308607 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:42.308566    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:43.018120 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:43.018083    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:43.091406 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:43.091370    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:43.151035 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:43.150999    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-2sqh9" event={"ID":"44adf4d8-cfe8-4db9-8c4d-ac493a58493b","Type":"ContainerStarted","Data":"2a27edf3d056932fc5ed188eff6ebc0e37c560a5814b6a69dbca29154b7dcc79"}
Jul 15 16:17:43.153735 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:43.153704    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hkgqt" event={"ID":"7a38e9b7-99c1-4867-a5c4-3cb9dead76be","Type":"ContainerStarted","Data":"2a59b7d43fc4ff581870885cdecd6690a20d4eb4470a4a6577fbc930f4b1803f"}
Jul 15 16:17:43.155312 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:43.155284    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7g2q8" event={"ID":"2241b2de-66eb-497c-b71c-34b725a04409","Type":"ContainerStarted","Data":"af26e6d8f2175d77c2abb626a1eaf9d6425456c4adb172212d2f63df48d64f96"}
Jul 15 16:17:43.156991 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:43.156965    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-2z6kg" event={"ID":"7ab4d8ae-d58c-4d06-b208-73a036fd4f69","Type":"ContainerStarted","Data":"da148f0c605a96fb8e54062fbde7b6e551ccddc880c17f245c0748746e9f3e32"}
Jul 15 16:17:43.177977 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:43.177581    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hkgqt" podStartSLOduration=2.521269465 podStartE2EDuration="16.17756097s" totalImagesPullingTime="13.656291505s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:17:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:17:43.176672472 +0000 UTC m=+64.823713008" watchObservedRunningTime="2026-07-15 16:17:43.17756097 +0000 UTC m=+64.824601706"
Jul 15 16:17:43.217312 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:43.217286    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:43.317216 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:43.317184    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:43.418544 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:43.418515    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:43.632358 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:43.632249    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:44.162699 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:44.162672    2580 generic.go:356] "Generic (PLEG): container finished" podID="2241b2de-66eb-497c-b71c-34b725a04409" containerID="0ea126f555740e2208cd6ae4e4f6eb3879e7da9db32028ee1da75ec0bcd405b2" exitCode=0
Jul 15 16:17:44.162826 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:44.162766    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7g2q8" event={"ID":"2241b2de-66eb-497c-b71c-34b725a04409","Type":"ContainerDied","Data":"0ea126f555740e2208cd6ae4e4f6eb3879e7da9db32028ee1da75ec0bcd405b2"}
Jul 15 16:17:44.627589 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:44.627543    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-metrics-certs\") pod \"network-metrics-daemon-g9k24\" (UID: \"a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd\") " pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:17:44.628096 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:44.627650    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7549f9c6-5988-4771-ae95-99e674252273-original-pull-secret\") pod \"global-pull-secret-syncer-c75b9\" (UID: \"7549f9c6-5988-4771-ae95-99e674252273\") " pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:17:44.630231 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:44.630205    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 16:17:44.630370 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:44.630208    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 16:17:44.641232 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:44.641198    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7549f9c6-5988-4771-ae95-99e674252273-original-pull-secret\") pod \"global-pull-secret-syncer-c75b9\" (UID: \"7549f9c6-5988-4771-ae95-99e674252273\") " pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:17:44.641347 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:44.641239    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd-metrics-certs\") pod \"network-metrics-daemon-g9k24\" (UID: \"a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd\") " pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:17:44.728729 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:44.728691    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85rfm\" (UniqueName: \"kubernetes.io/projected/422a9314-d5df-4554-ae6d-2a21856dcb6c-kube-api-access-85rfm\") pod \"network-check-target-t6fc5\" (UID: \"422a9314-d5df-4554-ae6d-2a21856dcb6c\") " pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:17:44.731574 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:44.731551    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 16:17:44.742474 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:44.742452    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 16:17:44.753240 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:44.753210    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85rfm\" (UniqueName: \"kubernetes.io/projected/422a9314-d5df-4554-ae6d-2a21856dcb6c-kube-api-access-85rfm\") pod \"network-check-target-t6fc5\" (UID: \"422a9314-d5df-4554-ae6d-2a21856dcb6c\") " pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:17:44.795473 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:44.795441    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c75b9"
Jul 15 16:17:44.807705 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:44.807670    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dmv8w\""
Jul 15 16:17:44.815338 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:44.815311    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:17:44.816071 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:44.815888    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wsr6k\""
Jul 15 16:17:44.823436 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:44.823415    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g9k24"
Jul 15 16:17:44.888517 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:44.888303    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:44.976686 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:44.976547    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c75b9"]
Jul 15 16:17:44.979966 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:17:44.979930    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7549f9c6_5988_4771_ae95_99e674252273.slice/crio-d279cd6996163655792b39fc425e53cfd6e5a2dc762621dd8fbb53a530760d22 WatchSource:0}: Error finding container d279cd6996163655792b39fc425e53cfd6e5a2dc762621dd8fbb53a530760d22: Status 404 returned error can't find the container with id d279cd6996163655792b39fc425e53cfd6e5a2dc762621dd8fbb53a530760d22
Jul 15 16:17:44.994708 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:44.994679    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:45.001125 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:45.001099    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-t6fc5"]
Jul 15 16:17:45.005227 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:17:45.005199    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod422a9314_d5df_4554_ae6d_2a21856dcb6c.slice/crio-9bd044eeaf78d699feded6272927777a9536a9500654ec7030976759ee2af064 WatchSource:0}: Error finding container 9bd044eeaf78d699feded6272927777a9536a9500654ec7030976759ee2af064: Status 404 returned error can't find the container with id 9bd044eeaf78d699feded6272927777a9536a9500654ec7030976759ee2af064
Jul 15 16:17:45.014099 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:45.014053    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-g9k24"]
Jul 15 16:17:45.018559 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:17:45.018531    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7cd60bc_9bdd_42f5_a60b_fdbbd4005cfd.slice/crio-a67d18d0d79cbee6ac56b29623e95fc6738f7cb205ae9007776a9258e29663e0 WatchSource:0}: Error finding container a67d18d0d79cbee6ac56b29623e95fc6738f7cb205ae9007776a9258e29663e0: Status 404 returned error can't find the container with id a67d18d0d79cbee6ac56b29623e95fc6738f7cb205ae9007776a9258e29663e0
Jul 15 16:17:45.092972 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:45.092935    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:45.123595 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:45.123565    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mng27"
Jul 15 16:17:45.168941 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:45.168784    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c75b9" event={"ID":"7549f9c6-5988-4771-ae95-99e674252273","Type":"ContainerStarted","Data":"d279cd6996163655792b39fc425e53cfd6e5a2dc762621dd8fbb53a530760d22"}
Jul 15 16:17:45.172102 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:45.172069    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7g2q8" event={"ID":"2241b2de-66eb-497c-b71c-34b725a04409","Type":"ContainerStarted","Data":"6c28be9991be2c5ee2f61b241283ee959d5f6dad901e0e456a88ba3b00cf1434"}
Jul 15 16:17:45.172250 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:45.172126    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7g2q8" event={"ID":"2241b2de-66eb-497c-b71c-34b725a04409","Type":"ContainerStarted","Data":"a19e68658b042556a321402737aa3aec97b9767dcf4bd4dd692d946fe3992a25"}
Jul 15 16:17:45.175169 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:45.175136    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-2z6kg" event={"ID":"7ab4d8ae-d58c-4d06-b208-73a036fd4f69","Type":"ContainerStarted","Data":"30261c1e18f631d069e8dfe29c796fbb11688a6867ff277654e78f5e882e6dc7"}
Jul 15 16:17:45.175295 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:45.175194    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-2z6kg" event={"ID":"7ab4d8ae-d58c-4d06-b208-73a036fd4f69","Type":"ContainerStarted","Data":"6207b392edcc654e778f06e8dd5b71c589dfc639cee04109c877ea2ea2c5ee1d"}
Jul 15 16:17:45.175295 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:45.175224    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-2z6kg" event={"ID":"7ab4d8ae-d58c-4d06-b208-73a036fd4f69","Type":"ContainerStarted","Data":"f7eb5801a004821d5f4165afef1d0d74e2fae252e54b8595139716a5eae943c2"}
Jul 15 16:17:45.176548 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:45.176511    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g9k24" event={"ID":"a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd","Type":"ContainerStarted","Data":"a67d18d0d79cbee6ac56b29623e95fc6738f7cb205ae9007776a9258e29663e0"}
Jul 15 16:17:45.178082 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:45.178048    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-t6fc5" event={"ID":"422a9314-d5df-4554-ae6d-2a21856dcb6c","Type":"ContainerStarted","Data":"9bd044eeaf78d699feded6272927777a9536a9500654ec7030976759ee2af064"}
Jul 15 16:17:45.196090 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:45.195436    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7g2q8" podStartSLOduration=4.138231995 podStartE2EDuration="5.195418078s" totalImagesPullingTime="983.110355ms" totalInitContainerRuntime="74.075728ms" isStatefulPod=true podCreationTimestamp="2026-07-15 16:17:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:17:45.194399111 +0000 UTC m=+66.841439647" watchObservedRunningTime="2026-07-15 16:17:45.195418078 +0000 UTC m=+66.842458594"
Jul 15 16:17:45.220172 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:45.220124    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-2z6kg" podStartSLOduration=3.311829225 podStartE2EDuration="5.220104592s" totalImagesPullingTime="1.908275367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:17:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:17:45.219480167 +0000 UTC m=+66.866520703" watchObservedRunningTime="2026-07-15 16:17:45.220104592 +0000 UTC m=+66.867145119"
Jul 15 16:17:46.638223 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.638060    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:46.645381 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.645351    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 16:17:46.669864 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.668687    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 16:17:46.669864 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.668935    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.675228 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.673260    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-7h469\""
Jul 15 16:17:46.675228 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.673707    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 15 16:17:46.675228 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.673811    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 15 16:17:46.675228 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.673815    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 15 16:17:46.675228 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.673921    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 15 16:17:46.675228 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.674242    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 15 16:17:46.675228 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.674295    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 15 16:17:46.675228 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.674463    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-eqkkqu00fv66k\""
Jul 15 16:17:46.675228 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.674534    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 15 16:17:46.675228 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.674463    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 15 16:17:46.675228 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.674732    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 15 16:17:46.675228 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.674914    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 15 16:17:46.675228 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.675121    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 15 16:17:46.675972 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.675252    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 15 16:17:46.675972 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.675396    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 15 16:17:46.677313 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.677289    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 15 16:17:46.681150 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.681134    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 15 16:17:46.745818 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.745774    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.745995 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.745833    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-web-config\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.745995 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.745874    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7krx7\" (UniqueName: \"kubernetes.io/projected/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-kube-api-access-7krx7\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.745995 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.745908    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.745995 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.745935    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.746219 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.745996    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.746219 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.746019    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.746219 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.746042    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.746219 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.746067    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.746219 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.746095    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.746219 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.746121    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.746219 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.746170    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-config-out\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.746219 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.746211    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.746598 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.746247    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.746598 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.746266    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.746598 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.746295    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-config\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.746598 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.746329    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.746598 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.746377    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.746598 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.746409    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.746598 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.746451    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.847138 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.847090    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.847295 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.847150    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.847295 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.847188    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-config-out\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.847295 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.847225    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.847468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.847341    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.847468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.847382    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.847468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.847411    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-config\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.847468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.847437    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.847659 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.847464    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.847659 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.847492    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.847659 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.847522    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.847659 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.847560    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.847659 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.847601    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-web-config\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.847927 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.847678    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7krx7\" (UniqueName: \"kubernetes.io/projected/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-kube-api-access-7krx7\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.847927 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.847709    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.847927 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.847741    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.847927 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.847794    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.847927 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.847830    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.847927 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.847852    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.847927 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.847877    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.848255 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.848028    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.848553 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.848525    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.848904 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.848884    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.849284 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.848974    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.849284 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.849070    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.852376 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.852353    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.852808 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.852785    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.853791 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.853749    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.854309 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.854283    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.854405 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.854289    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.854952 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.854925    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.855345 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.855320    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-web-config\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.856110 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.856059    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-config\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.856578 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.856539    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-config-out\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.857712 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.857638    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.858362 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.858336    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.858950 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.858883    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.861497 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.861420    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.861497 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.861486    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7krx7\" (UniqueName: \"kubernetes.io/projected/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-kube-api-access-7krx7\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.862909 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.862869    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:46.984549 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:46.984464    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:17:47.190826 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:47.190740    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g9k24" event={"ID":"a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd","Type":"ContainerStarted","Data":"53fd274214b1e00efa9a3ea4113a5e1c00b0540462fed624b55f2b2359bf25e1"}
Jul 15 16:17:47.190826 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:47.190831    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g9k24" event={"ID":"a7cd60bc-9bdd-42f5-a60b-fdbbd4005cfd","Type":"ContainerStarted","Data":"948e9522bb0cd7df899d3b96fc707ba3330efb005a3724e1b22df12024d5b71b"}
Jul 15 16:17:47.193043 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:47.192936    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 16:17:47.199744 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:17:47.199714    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f4a5e3c_b014_463a_96ff_3e2dbdbdf8c6.slice/crio-fdcda00085b488247182ac1a795525f4a9ea766b9ca52d28edb4ec1c244fb973 WatchSource:0}: Error finding container fdcda00085b488247182ac1a795525f4a9ea766b9ca52d28edb4ec1c244fb973: Status 404 returned error can't find the container with id fdcda00085b488247182ac1a795525f4a9ea766b9ca52d28edb4ec1c244fb973
Jul 15 16:17:47.211323 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:47.211268    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-g9k24" podStartSLOduration=66.891006747 podStartE2EDuration="1m8.211246619s" totalImagesPullingTime="1.320239872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:17:47.210907489 +0000 UTC m=+68.857948026" watchObservedRunningTime="2026-07-15 16:17:47.211246619 +0000 UTC m=+68.858287159"
Jul 15 16:17:47.342771 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:47.342711    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:47.443940 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:47.443895    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:47.803243 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:47.803176    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:47.881714 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:47.881642    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:48.196907 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:48.196802    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6","Type":"ContainerStarted","Data":"fdcda00085b488247182ac1a795525f4a9ea766b9ca52d28edb4ec1c244fb973"}
Jul 15 16:17:48.206672 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:48.206638    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:48.303843 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:48.303742    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:51.105402 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:51.105371    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-59798f98df-72z7k"
Jul 15 16:17:51.279116 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:51.279054    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:51.377423 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:51.377383    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:52.219687 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:52.219645    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c75b9" event={"ID":"7549f9c6-5988-4771-ae95-99e674252273","Type":"ContainerStarted","Data":"d257f2e38b662b96b6b47b5713db2e740b3a326dbb291f15d7db5ea4d813f995"}
Jul 15 16:17:52.221186 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:52.221161    2580 generic.go:356] "Generic (PLEG): container finished" podID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerID="b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da" exitCode=0
Jul 15 16:17:52.221298 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:52.221246    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6","Type":"ContainerDied","Data":"b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da"}
Jul 15 16:17:52.222623 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:52.222585    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-t6fc5" event={"ID":"422a9314-d5df-4554-ae6d-2a21856dcb6c","Type":"ContainerStarted","Data":"db53f46486e94ebe3a0833500a3453603d69f1896d503a796ffca41ece0d6063"}
Jul 15 16:17:52.222779 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:52.222732    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:17:52.236105 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:52.236064    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-c75b9" podStartSLOduration=66.949309575 podStartE2EDuration="1m13.236048355s" totalImagesPullingTime="6.28673878s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:17:52.235293809 +0000 UTC m=+73.882334344" watchObservedRunningTime="2026-07-15 16:17:52.236048355 +0000 UTC m=+73.883088892"
Jul 15 16:17:52.252518 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:52.252481    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-t6fc5" podStartSLOduration=66.998227724 podStartE2EDuration="1m13.252468541s" totalImagesPullingTime="6.254240817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:17:52.251673943 +0000 UTC m=+73.898714480" watchObservedRunningTime="2026-07-15 16:17:52.252468541 +0000 UTC m=+73.899509077"
Jul 15 16:17:52.907110 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:52.907076    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:52.974339 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:52.974300    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:54.029638 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:54.029599    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:54.431901 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:54.431866    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:54.496102 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:54.496071    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:56.643039 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:56.643006    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:56.779499 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:56.779457    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:59.463854 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:59.463802    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:17:59.555224 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:17:59.555179    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:18:02.492063 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:02.491911    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:18:02.784473 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:02.784376    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:18:02.842048 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:02.842005    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:18:04.270550 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:04.270511    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-2sqh9" event={"ID":"44adf4d8-cfe8-4db9-8c4d-ac493a58493b","Type":"ContainerStarted","Data":"5858e0dfa2d516d16c44e1843fec700365c1914a85ababa8461280b510f0d850"}
Jul 15 16:18:04.271061 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:04.270689    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-2sqh9"
Jul 15 16:18:04.273810 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:04.273784    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6","Type":"ContainerStarted","Data":"1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10"}
Jul 15 16:18:04.273928 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:04.273839    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6","Type":"ContainerStarted","Data":"6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1"}
Jul 15 16:18:04.290009 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:04.289140    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-2sqh9"
Jul 15 16:18:04.290462 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:04.290402    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-2sqh9" podStartSLOduration=2.971012912 podStartE2EDuration="24.290382969s" totalImagesPullingTime="21.319370057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:17:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:18:04.290100901 +0000 UTC m=+85.937141463" watchObservedRunningTime="2026-07-15 16:18:04.290382969 +0000 UTC m=+85.937423507"
Jul 15 16:18:04.822987 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:04.822914    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:18:04.934804 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:04.934667    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:18:05.045070 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:05.045031    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:18:06.882503 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:06.882427    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:18:07.030515 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:07.030458    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:18:07.132123 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:07.132039    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:18:07.232272 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:07.232239    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:18:07.290553 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:07.290515    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6","Type":"ContainerStarted","Data":"656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410"}
Jul 15 16:18:07.290737 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:07.290647    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6","Type":"ContainerStarted","Data":"889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467"}
Jul 15 16:18:07.290737 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:07.290695    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6","Type":"ContainerStarted","Data":"d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98"}
Jul 15 16:18:07.290884 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:07.290743    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6","Type":"ContainerStarted","Data":"a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d"}
Jul 15 16:18:07.326351 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:07.326286    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.10523956 podStartE2EDuration="21.326263236s" totalImagesPullingTime="18.197314717s" totalInitContainerRuntime="23.708959ms" isStatefulPod=false podCreationTimestamp="2026-07-15 16:17:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:18:07.323648073 +0000 UTC m=+88.970688659" watchObservedRunningTime="2026-07-15 16:18:07.326263236 +0000 UTC m=+88.973303773"
Jul 15 16:18:11.984645 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:11.984614    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:18:17.845155 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:17.845099    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:18:17.923059 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:17.923030    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:18:23.230342 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:23.230308    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-t6fc5"
Jul 15 16:18:46.984860 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:46.984810    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:18:47.042529 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:47.042501    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:18:47.453579 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:47.453544    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:18:59.593931 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:59.593893    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:18:59.662230 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:18:59.662195    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:19:04.936822 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:04.936786    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 16:19:04.937251 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:04.937100    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="prometheus" containerID="cri-o://6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1" gracePeriod=600
Jul 15 16:19:04.937251 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:04.937109    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="kube-rbac-proxy-thanos" containerID="cri-o://656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410" gracePeriod=600
Jul 15 16:19:04.937251 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:04.937134    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="thanos-sidecar" containerID="cri-o://a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d" gracePeriod=600
Jul 15 16:19:04.937251 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:04.937148    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="kube-rbac-proxy-web" containerID="cri-o://d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98" gracePeriod=600
Jul 15 16:19:04.937251 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:04.937185    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="kube-rbac-proxy" containerID="cri-o://889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467" gracePeriod=600
Jul 15 16:19:04.937251 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:04.937209    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="config-reloader" containerID="cri-o://1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10" gracePeriod=600
Jul 15 16:19:05.166270 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.166237    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.280414 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.280326    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-config-out\") pod \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") "
Jul 15 16:19:05.280414 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.280376    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-configmap-kubelet-serving-ca-bundle\") pod \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") "
Jul 15 16:19:05.280414 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.280410    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7krx7\" (UniqueName: \"kubernetes.io/projected/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-kube-api-access-7krx7\") pod \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") "
Jul 15 16:19:05.280655 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.280445    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-configmap-metrics-client-ca\") pod \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") "
Jul 15 16:19:05.280655 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.280472    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-tls-assets\") pod \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") "
Jul 15 16:19:05.280655 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.280498    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-config\") pod \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") "
Jul 15 16:19:05.280655 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.280521    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-k8s-rulefiles-0\") pod \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") "
Jul 15 16:19:05.280907 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.280669    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-k8s-rulefiles-2\") pod \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") "
Jul 15 16:19:05.280907 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.280716    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-k8s-rulefiles-1\") pod \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") "
Jul 15 16:19:05.280907 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.280782    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-web-config\") pod \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") "
Jul 15 16:19:05.280907 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.280810    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-k8s-db\") pod \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") "
Jul 15 16:19:05.280907 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.280847    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-trusted-ca-bundle\") pod \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") "
Jul 15 16:19:05.280907 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.280889    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") "
Jul 15 16:19:05.281242 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.280915    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-configmap-serving-certs-ca-bundle\") pod \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") "
Jul 15 16:19:05.281242 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.280977    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-thanos-prometheus-http-client-file\") pod \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") "
Jul 15 16:19:05.281242 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.280993    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" (UID: "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:19:05.281242 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.281006    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-grpc-tls\") pod \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") "
Jul 15 16:19:05.281242 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.281084    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-metrics-client-certs\") pod \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") "
Jul 15 16:19:05.281242 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.281115    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-kube-rbac-proxy\") pod \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") "
Jul 15 16:19:05.281242 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.281142    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-prometheus-k8s-tls\") pod \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") "
Jul 15 16:19:05.281242 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.281174    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\" (UID: \"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6\") "
Jul 15 16:19:05.281608 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.281402    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" (UID: "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:19:05.281608 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.281411    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-configmap-metrics-client-ca\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:19:05.282854 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.282008    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" (UID: "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:19:05.282854 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.282047    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" (UID: "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:19:05.282854 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.282417    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" (UID: "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:19:05.282854 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.282442    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" (UID: "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:19:05.282854 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.282598    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" (UID: "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:19:05.282854 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.282823    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" (UID: "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:19:05.283460 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.283436    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-config" (OuterVolumeSpecName: "config") pod "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" (UID: "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:19:05.283541 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.283498    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-config-out" (OuterVolumeSpecName: "config-out") pod "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" (UID: "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:19:05.284026 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.283950    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-kube-api-access-7krx7" (OuterVolumeSpecName: "kube-api-access-7krx7") pod "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" (UID: "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6"). InnerVolumeSpecName "kube-api-access-7krx7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:19:05.284026 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.283951    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" (UID: "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:19:05.284380 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.284353    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" (UID: "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:19:05.285227 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.285198    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" (UID: "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:19:05.285590 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.285568    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" (UID: "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:19:05.285960 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.285942    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" (UID: "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:19:05.286184 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.286165    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" (UID: "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:19:05.286268 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.286248    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" (UID: "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:19:05.286581 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.286563    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" (UID: "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:19:05.294598 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.294577    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-web-config" (OuterVolumeSpecName: "web-config") pod "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" (UID: "2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:19:05.382096 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.382053    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:19:05.382096 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.382089    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:19:05.382096 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.382104    2580 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-thanos-prometheus-http-client-file\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:19:05.382364 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.382116    2580 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-grpc-tls\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:19:05.382364 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.382129    2580 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-metrics-client-certs\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:19:05.382364 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.382141    2580 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-kube-rbac-proxy\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:19:05.382364 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.382152    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-prometheus-k8s-tls\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:19:05.382364 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.382164    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:19:05.382364 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.382175    2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-config-out\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:19:05.382364 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.382186    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:19:05.382364 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.382197    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7krx7\" (UniqueName: \"kubernetes.io/projected/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-kube-api-access-7krx7\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:19:05.382364 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.382208    2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-tls-assets\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:19:05.382364 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.382219    2580 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-config\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:19:05.382364 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.382230    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:19:05.382364 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.382243    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:19:05.382364 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.382256    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:19:05.382364 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.382268    2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-web-config\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:19:05.382364 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.382279    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-k8s-db\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:19:05.382364 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.382292    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6-prometheus-trusted-ca-bundle\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:19:05.499044 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.499013    2580 generic.go:356] "Generic (PLEG): container finished" podID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerID="656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410" exitCode=0
Jul 15 16:19:05.499044 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.499036    2580 generic.go:356] "Generic (PLEG): container finished" podID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerID="889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467" exitCode=0
Jul 15 16:19:05.499044 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.499042    2580 generic.go:356] "Generic (PLEG): container finished" podID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerID="d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98" exitCode=0
Jul 15 16:19:05.499044 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.499048    2580 generic.go:356] "Generic (PLEG): container finished" podID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerID="a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d" exitCode=0
Jul 15 16:19:05.499044 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.499053    2580 generic.go:356] "Generic (PLEG): container finished" podID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerID="1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10" exitCode=0
Jul 15 16:19:05.499418 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.499058    2580 generic.go:356] "Generic (PLEG): container finished" podID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerID="6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1" exitCode=0
Jul 15 16:19:05.499418 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.499098    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6","Type":"ContainerDied","Data":"656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410"}
Jul 15 16:19:05.499418 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.499137    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.499418 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.499169    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6","Type":"ContainerDied","Data":"889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467"}
Jul 15 16:19:05.499418 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.499197    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6","Type":"ContainerDied","Data":"d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98"}
Jul 15 16:19:05.499418 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.499222    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6","Type":"ContainerDied","Data":"a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d"}
Jul 15 16:19:05.499418 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.499237    2580 scope.go:117] "RemoveContainer" containerID="656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410"
Jul 15 16:19:05.499418 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.499256    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6","Type":"ContainerDied","Data":"1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10"}
Jul 15 16:19:05.499418 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.499280    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6","Type":"ContainerDied","Data":"6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1"}
Jul 15 16:19:05.499418 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.499311    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6","Type":"ContainerDied","Data":"fdcda00085b488247182ac1a795525f4a9ea766b9ca52d28edb4ec1c244fb973"}
Jul 15 16:19:05.507860 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.507841    2580 scope.go:117] "RemoveContainer" containerID="889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467"
Jul 15 16:19:05.514466 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.514449    2580 scope.go:117] "RemoveContainer" containerID="d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98"
Jul 15 16:19:05.523843 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.523810    2580 scope.go:117] "RemoveContainer" containerID="a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d"
Jul 15 16:19:05.524104 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.524075    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 16:19:05.528198 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.528177    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 16:19:05.530628 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.530612    2580 scope.go:117] "RemoveContainer" containerID="1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10"
Jul 15 16:19:05.536948 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.536931    2580 scope.go:117] "RemoveContainer" containerID="6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1"
Jul 15 16:19:05.543498 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.543482    2580 scope.go:117] "RemoveContainer" containerID="b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da"
Jul 15 16:19:05.549715 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.549699    2580 scope.go:117] "RemoveContainer" containerID="656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410"
Jul 15 16:19:05.550000 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:19:05.549979    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410\": container with ID starting with 656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410 not found: ID does not exist" containerID="656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410"
Jul 15 16:19:05.550060 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.550010    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410"} err="failed to get container status \"656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410\": rpc error: code = NotFound desc = could not find container \"656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410\": container with ID starting with 656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410 not found: ID does not exist"
Jul 15 16:19:05.550060 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.550048    2580 scope.go:117] "RemoveContainer" containerID="889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467"
Jul 15 16:19:05.550288 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:19:05.550270    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467\": container with ID starting with 889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467 not found: ID does not exist" containerID="889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467"
Jul 15 16:19:05.550332 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.550297    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467"} err="failed to get container status \"889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467\": rpc error: code = NotFound desc = could not find container \"889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467\": container with ID starting with 889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467 not found: ID does not exist"
Jul 15 16:19:05.550332 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.550315    2580 scope.go:117] "RemoveContainer" containerID="d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98"
Jul 15 16:19:05.550520 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:19:05.550504    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98\": container with ID starting with d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98 not found: ID does not exist" containerID="d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98"
Jul 15 16:19:05.550566 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.550526    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98"} err="failed to get container status \"d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98\": rpc error: code = NotFound desc = could not find container \"d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98\": container with ID starting with d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98 not found: ID does not exist"
Jul 15 16:19:05.550566 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.550541    2580 scope.go:117] "RemoveContainer" containerID="a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d"
Jul 15 16:19:05.550751 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:19:05.550737    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d\": container with ID starting with a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d not found: ID does not exist" containerID="a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d"
Jul 15 16:19:05.550809 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.550769    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d"} err="failed to get container status \"a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d\": rpc error: code = NotFound desc = could not find container \"a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d\": container with ID starting with a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d not found: ID does not exist"
Jul 15 16:19:05.550809 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.550788    2580 scope.go:117] "RemoveContainer" containerID="1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10"
Jul 15 16:19:05.551024 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:19:05.551010    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10\": container with ID starting with 1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10 not found: ID does not exist" containerID="1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10"
Jul 15 16:19:05.551061 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.551027    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10"} err="failed to get container status \"1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10\": rpc error: code = NotFound desc = could not find container \"1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10\": container with ID starting with 1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10 not found: ID does not exist"
Jul 15 16:19:05.551061 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.551039    2580 scope.go:117] "RemoveContainer" containerID="6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1"
Jul 15 16:19:05.551211 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:19:05.551192    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1\": container with ID starting with 6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1 not found: ID does not exist" containerID="6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1"
Jul 15 16:19:05.551252 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.551219    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1"} err="failed to get container status \"6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1\": rpc error: code = NotFound desc = could not find container \"6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1\": container with ID starting with 6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1 not found: ID does not exist"
Jul 15 16:19:05.551252 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.551241    2580 scope.go:117] "RemoveContainer" containerID="b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da"
Jul 15 16:19:05.551463 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:19:05.551449    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da\": container with ID starting with b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da not found: ID does not exist" containerID="b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da"
Jul 15 16:19:05.551506 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.551466    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da"} err="failed to get container status \"b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da\": rpc error: code = NotFound desc = could not find container \"b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da\": container with ID starting with b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da not found: ID does not exist"
Jul 15 16:19:05.551506 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.551478    2580 scope.go:117] "RemoveContainer" containerID="656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410"
Jul 15 16:19:05.551695 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.551674    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410"} err="failed to get container status \"656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410\": rpc error: code = NotFound desc = could not find container \"656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410\": container with ID starting with 656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410 not found: ID does not exist"
Jul 15 16:19:05.551736 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.551696    2580 scope.go:117] "RemoveContainer" containerID="889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467"
Jul 15 16:19:05.551903 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.551887    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467"} err="failed to get container status \"889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467\": rpc error: code = NotFound desc = could not find container \"889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467\": container with ID starting with 889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467 not found: ID does not exist"
Jul 15 16:19:05.551948 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.551903    2580 scope.go:117] "RemoveContainer" containerID="d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98"
Jul 15 16:19:05.552106 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.552089    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98"} err="failed to get container status \"d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98\": rpc error: code = NotFound desc = could not find container \"d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98\": container with ID starting with d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98 not found: ID does not exist"
Jul 15 16:19:05.552153 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.552107    2580 scope.go:117] "RemoveContainer" containerID="a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d"
Jul 15 16:19:05.552320 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.552301    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d"} err="failed to get container status \"a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d\": rpc error: code = NotFound desc = could not find container \"a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d\": container with ID starting with a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d not found: ID does not exist"
Jul 15 16:19:05.552395 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.552322    2580 scope.go:117] "RemoveContainer" containerID="1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10"
Jul 15 16:19:05.552531 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.552513    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10"} err="failed to get container status \"1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10\": rpc error: code = NotFound desc = could not find container \"1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10\": container with ID starting with 1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10 not found: ID does not exist"
Jul 15 16:19:05.552570 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.552531    2580 scope.go:117] "RemoveContainer" containerID="6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1"
Jul 15 16:19:05.552698 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.552682    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1"} err="failed to get container status \"6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1\": rpc error: code = NotFound desc = could not find container \"6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1\": container with ID starting with 6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1 not found: ID does not exist"
Jul 15 16:19:05.552738 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.552698    2580 scope.go:117] "RemoveContainer" containerID="b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da"
Jul 15 16:19:05.552919 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.552902    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da"} err="failed to get container status \"b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da\": rpc error: code = NotFound desc = could not find container \"b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da\": container with ID starting with b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da not found: ID does not exist"
Jul 15 16:19:05.552979 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.552920    2580 scope.go:117] "RemoveContainer" containerID="656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410"
Jul 15 16:19:05.553085 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.553069    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410"} err="failed to get container status \"656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410\": rpc error: code = NotFound desc = could not find container \"656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410\": container with ID starting with 656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410 not found: ID does not exist"
Jul 15 16:19:05.553124 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.553085    2580 scope.go:117] "RemoveContainer" containerID="889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467"
Jul 15 16:19:05.553263 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.553248    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467"} err="failed to get container status \"889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467\": rpc error: code = NotFound desc = could not find container \"889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467\": container with ID starting with 889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467 not found: ID does not exist"
Jul 15 16:19:05.553308 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.553263    2580 scope.go:117] "RemoveContainer" containerID="d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98"
Jul 15 16:19:05.553463 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.553445    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98"} err="failed to get container status \"d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98\": rpc error: code = NotFound desc = could not find container \"d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98\": container with ID starting with d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98 not found: ID does not exist"
Jul 15 16:19:05.553508 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.553464    2580 scope.go:117] "RemoveContainer" containerID="a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d"
Jul 15 16:19:05.553681 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.553663    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d"} err="failed to get container status \"a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d\": rpc error: code = NotFound desc = could not find container \"a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d\": container with ID starting with a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d not found: ID does not exist"
Jul 15 16:19:05.553745 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.553682    2580 scope.go:117] "RemoveContainer" containerID="1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10"
Jul 15 16:19:05.553930 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.553910    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10"} err="failed to get container status \"1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10\": rpc error: code = NotFound desc = could not find container \"1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10\": container with ID starting with 1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10 not found: ID does not exist"
Jul 15 16:19:05.553977 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.553931    2580 scope.go:117] "RemoveContainer" containerID="6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1"
Jul 15 16:19:05.554140 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.554124    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1"} err="failed to get container status \"6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1\": rpc error: code = NotFound desc = could not find container \"6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1\": container with ID starting with 6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1 not found: ID does not exist"
Jul 15 16:19:05.554184 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.554141    2580 scope.go:117] "RemoveContainer" containerID="b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da"
Jul 15 16:19:05.554345 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.554328    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da"} err="failed to get container status \"b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da\": rpc error: code = NotFound desc = could not find container \"b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da\": container with ID starting with b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da not found: ID does not exist"
Jul 15 16:19:05.554409 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.554347    2580 scope.go:117] "RemoveContainer" containerID="656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410"
Jul 15 16:19:05.554550 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.554528    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410"} err="failed to get container status \"656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410\": rpc error: code = NotFound desc = could not find container \"656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410\": container with ID starting with 656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410 not found: ID does not exist"
Jul 15 16:19:05.554594 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.554550    2580 scope.go:117] "RemoveContainer" containerID="889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467"
Jul 15 16:19:05.554728 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.554710    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467"} err="failed to get container status \"889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467\": rpc error: code = NotFound desc = could not find container \"889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467\": container with ID starting with 889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467 not found: ID does not exist"
Jul 15 16:19:05.554796 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.554729    2580 scope.go:117] "RemoveContainer" containerID="d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98"
Jul 15 16:19:05.554957 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.554937    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98"} err="failed to get container status \"d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98\": rpc error: code = NotFound desc = could not find container \"d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98\": container with ID starting with d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98 not found: ID does not exist"
Jul 15 16:19:05.555024 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.554958    2580 scope.go:117] "RemoveContainer" containerID="a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d"
Jul 15 16:19:05.555183 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.555165    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d"} err="failed to get container status \"a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d\": rpc error: code = NotFound desc = could not find container \"a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d\": container with ID starting with a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d not found: ID does not exist"
Jul 15 16:19:05.555234 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.555184    2580 scope.go:117] "RemoveContainer" containerID="1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10"
Jul 15 16:19:05.555367 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.555351    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10"} err="failed to get container status \"1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10\": rpc error: code = NotFound desc = could not find container \"1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10\": container with ID starting with 1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10 not found: ID does not exist"
Jul 15 16:19:05.555408 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.555368    2580 scope.go:117] "RemoveContainer" containerID="6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1"
Jul 15 16:19:05.555542 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.555526    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1"} err="failed to get container status \"6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1\": rpc error: code = NotFound desc = could not find container \"6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1\": container with ID starting with 6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1 not found: ID does not exist"
Jul 15 16:19:05.555585 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.555542    2580 scope.go:117] "RemoveContainer" containerID="b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da"
Jul 15 16:19:05.555749 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.555733    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da"} err="failed to get container status \"b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da\": rpc error: code = NotFound desc = could not find container \"b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da\": container with ID starting with b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da not found: ID does not exist"
Jul 15 16:19:05.555749 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.555748    2580 scope.go:117] "RemoveContainer" containerID="656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410"
Jul 15 16:19:05.555967 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.555948    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410"} err="failed to get container status \"656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410\": rpc error: code = NotFound desc = could not find container \"656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410\": container with ID starting with 656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410 not found: ID does not exist"
Jul 15 16:19:05.556096 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.555968    2580 scope.go:117] "RemoveContainer" containerID="889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467"
Jul 15 16:19:05.556601 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.556231    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467"} err="failed to get container status \"889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467\": rpc error: code = NotFound desc = could not find container \"889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467\": container with ID starting with 889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467 not found: ID does not exist"
Jul 15 16:19:05.556601 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.556266    2580 scope.go:117] "RemoveContainer" containerID="d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98"
Jul 15 16:19:05.556917 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.556866    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98"} err="failed to get container status \"d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98\": rpc error: code = NotFound desc = could not find container \"d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98\": container with ID starting with d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98 not found: ID does not exist"
Jul 15 16:19:05.556917 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.556914    2580 scope.go:117] "RemoveContainer" containerID="a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d"
Jul 15 16:19:05.557196 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.557174    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d"} err="failed to get container status \"a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d\": rpc error: code = NotFound desc = could not find container \"a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d\": container with ID starting with a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d not found: ID does not exist"
Jul 15 16:19:05.557196 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.557195    2580 scope.go:117] "RemoveContainer" containerID="1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10"
Jul 15 16:19:05.557455 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.557425    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10"} err="failed to get container status \"1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10\": rpc error: code = NotFound desc = could not find container \"1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10\": container with ID starting with 1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10 not found: ID does not exist"
Jul 15 16:19:05.557455 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.557447    2580 scope.go:117] "RemoveContainer" containerID="6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1"
Jul 15 16:19:05.557716 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.557699    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1"} err="failed to get container status \"6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1\": rpc error: code = NotFound desc = could not find container \"6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1\": container with ID starting with 6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1 not found: ID does not exist"
Jul 15 16:19:05.557811 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.557719    2580 scope.go:117] "RemoveContainer" containerID="b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da"
Jul 15 16:19:05.557992 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.557970    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da"} err="failed to get container status \"b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da\": rpc error: code = NotFound desc = could not find container \"b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da\": container with ID starting with b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da not found: ID does not exist"
Jul 15 16:19:05.557992 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.557993    2580 scope.go:117] "RemoveContainer" containerID="656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410"
Jul 15 16:19:05.558131 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558073    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 16:19:05.558261 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558243    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410"} err="failed to get container status \"656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410\": rpc error: code = NotFound desc = could not find container \"656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410\": container with ID starting with 656c5b923de7f3aac6cca015af21d25c9ce09955357e83afc5d43aef8ef82410 not found: ID does not exist"
Jul 15 16:19:05.558310 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558263    2580 scope.go:117] "RemoveContainer" containerID="889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467"
Jul 15 16:19:05.558362 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558344    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="prometheus"
Jul 15 16:19:05.558362 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558364    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="prometheus"
Jul 15 16:19:05.558501 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558373    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="thanos-sidecar"
Jul 15 16:19:05.558501 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558381    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="thanos-sidecar"
Jul 15 16:19:05.558501 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558395    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="config-reloader"
Jul 15 16:19:05.558501 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558404    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="config-reloader"
Jul 15 16:19:05.558501 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558413    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="kube-rbac-proxy-thanos"
Jul 15 16:19:05.558501 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558420    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="kube-rbac-proxy-thanos"
Jul 15 16:19:05.558501 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558426    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="init-config-reloader"
Jul 15 16:19:05.558501 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558431    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="init-config-reloader"
Jul 15 16:19:05.558501 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558439    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="kube-rbac-proxy-web"
Jul 15 16:19:05.558501 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558447    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="kube-rbac-proxy-web"
Jul 15 16:19:05.558501 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558453    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="kube-rbac-proxy"
Jul 15 16:19:05.558501 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558458    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="kube-rbac-proxy"
Jul 15 16:19:05.558501 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558474    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467"} err="failed to get container status \"889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467\": rpc error: code = NotFound desc = could not find container \"889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467\": container with ID starting with 889434a00b10557a363857ffcf8747073dbd41b3bbc4c6e5b0a14f845cb4c467 not found: ID does not exist"
Jul 15 16:19:05.558501 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558494    2580 scope.go:117] "RemoveContainer" containerID="d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98"
Jul 15 16:19:05.558501 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558506    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="kube-rbac-proxy"
Jul 15 16:19:05.559088 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558517    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="prometheus"
Jul 15 16:19:05.559088 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558526    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="thanos-sidecar"
Jul 15 16:19:05.559088 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558534    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="kube-rbac-proxy-web"
Jul 15 16:19:05.559088 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558541    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="config-reloader"
Jul 15 16:19:05.559088 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558545    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" containerName="kube-rbac-proxy-thanos"
Jul 15 16:19:05.559088 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558689    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98"} err="failed to get container status \"d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98\": rpc error: code = NotFound desc = could not find container \"d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98\": container with ID starting with d7a3a4fbb900e50e27089e22896dc09f82123f851527defc7a55f1f7815a5d98 not found: ID does not exist"
Jul 15 16:19:05.559088 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558701    2580 scope.go:117] "RemoveContainer" containerID="a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d"
Jul 15 16:19:05.559088 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558968    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d"} err="failed to get container status \"a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d\": rpc error: code = NotFound desc = could not find container \"a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d\": container with ID starting with a00d44332ceba0ffc12fe21c13ecf062271158b749f8a1a7b7e8feca0bb3806d not found: ID does not exist"
Jul 15 16:19:05.559088 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.558993    2580 scope.go:117] "RemoveContainer" containerID="1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10"
Jul 15 16:19:05.559446 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.559248    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10"} err="failed to get container status \"1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10\": rpc error: code = NotFound desc = could not find container \"1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10\": container with ID starting with 1afa5de08d5157d668c566074ab0a2fa40657d6de0f38264b35399292a0fec10 not found: ID does not exist"
Jul 15 16:19:05.559446 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.559274    2580 scope.go:117] "RemoveContainer" containerID="6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1"
Jul 15 16:19:05.559521 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.559496    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1"} err="failed to get container status \"6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1\": rpc error: code = NotFound desc = could not find container \"6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1\": container with ID starting with 6ddda3a5d8c47f3ec5a5c1fa5bca3d2237464781c36aba6a519ca45a2c8bdee1 not found: ID does not exist"
Jul 15 16:19:05.559521 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.559511    2580 scope.go:117] "RemoveContainer" containerID="b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da"
Jul 15 16:19:05.559739 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.559720    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da"} err="failed to get container status \"b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da\": rpc error: code = NotFound desc = could not find container \"b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da\": container with ID starting with b00b457c66d5a9b33506dd48539d37eb21899975d1c8f4a9cbbc097993a919da not found: ID does not exist"
Jul 15 16:19:05.562398 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.562385    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.565986 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.565939    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 15 16:19:05.566070 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.565964    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 15 16:19:05.566070 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.565964    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 15 16:19:05.566070 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.566063    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 15 16:19:05.566234 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.566098    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 15 16:19:05.566234 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.566152    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 15 16:19:05.566234 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.566160    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-7h469\""
Jul 15 16:19:05.566234 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.566160    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 15 16:19:05.566234 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.566222    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 15 16:19:05.566459 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.566280    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 15 16:19:05.566459 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.566291    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 15 16:19:05.567257 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.567239    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-eqkkqu00fv66k\""
Jul 15 16:19:05.567500 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.567475    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 15 16:19:05.567712 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.567683    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 15 16:19:05.567832 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.567820    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 15 16:19:05.569835 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.569815    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 15 16:19:05.571928 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.571911    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 15 16:19:05.580110 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.580090    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 16:19:05.684852 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.684808    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7b65c1c2-a69a-444b-868e-2e8265cff8f3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.684852 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.684848    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b65c1c2-a69a-444b-868e-2e8265cff8f3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.685067 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.684868    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-config\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.685067 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.684919    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.685067 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.684950    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b27wc\" (UniqueName: \"kubernetes.io/projected/7b65c1c2-a69a-444b-868e-2e8265cff8f3-kube-api-access-b27wc\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.685067 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.684996    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b65c1c2-a69a-444b-868e-2e8265cff8f3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.685067 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.685025    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.685067 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.685061    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7b65c1c2-a69a-444b-868e-2e8265cff8f3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.685238 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.685081    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.685238 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.685109    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.685238 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.685124    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.685238 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.685153    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7b65c1c2-a69a-444b-868e-2e8265cff8f3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.685238 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.685181    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b65c1c2-a69a-444b-868e-2e8265cff8f3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.685238 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.685204    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-web-config\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.685238 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.685225    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7b65c1c2-a69a-444b-868e-2e8265cff8f3-config-out\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.685431 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.685249    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.685431 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.685266    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7b65c1c2-a69a-444b-868e-2e8265cff8f3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.685431 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.685294    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b65c1c2-a69a-444b-868e-2e8265cff8f3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.685431 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.685316    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7b65c1c2-a69a-444b-868e-2e8265cff8f3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.685431 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.685330    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.786137 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.786057    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.786137 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.786100    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.786278 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.786147    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.786278 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.786176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7b65c1c2-a69a-444b-868e-2e8265cff8f3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.786415 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.786390    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b65c1c2-a69a-444b-868e-2e8265cff8f3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.786486 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.786437    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-web-config\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.786486 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.786468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7b65c1c2-a69a-444b-868e-2e8265cff8f3-config-out\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.786584 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.786508    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.786632 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.786557    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7b65c1c2-a69a-444b-868e-2e8265cff8f3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.786632 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.786614    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b65c1c2-a69a-444b-868e-2e8265cff8f3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.786738 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.786641    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7b65c1c2-a69a-444b-868e-2e8265cff8f3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.786738 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.786665    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.786738 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.786689    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7b65c1c2-a69a-444b-868e-2e8265cff8f3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.786738 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.786707    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7b65c1c2-a69a-444b-868e-2e8265cff8f3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.787049 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.786737    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b65c1c2-a69a-444b-868e-2e8265cff8f3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.787049 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.786777    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-config\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.787049 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.786801    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.787049 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.786834    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b27wc\" (UniqueName: \"kubernetes.io/projected/7b65c1c2-a69a-444b-868e-2e8265cff8f3-kube-api-access-b27wc\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.787049 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.786870    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b65c1c2-a69a-444b-868e-2e8265cff8f3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.787049 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.786912    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.787049 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.786947    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7b65c1c2-a69a-444b-868e-2e8265cff8f3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.787390 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.787343    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b65c1c2-a69a-444b-868e-2e8265cff8f3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.789218 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.787343    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b65c1c2-a69a-444b-868e-2e8265cff8f3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.789400 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.789369    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.789880 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.789565    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7b65c1c2-a69a-444b-868e-2e8265cff8f3-config-out\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.789880 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.789612    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.790050 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.789947    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7b65c1c2-a69a-444b-868e-2e8265cff8f3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.790103 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.790054    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.790103 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.790093    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.790206 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.790118    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7b65c1c2-a69a-444b-868e-2e8265cff8f3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.790576 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.790532    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b65c1c2-a69a-444b-868e-2e8265cff8f3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.790794 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.790727    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-web-config\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.790920 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.790898    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b65c1c2-a69a-444b-868e-2e8265cff8f3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.791234 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.791214    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7b65c1c2-a69a-444b-868e-2e8265cff8f3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.791411 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.791396    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.791604 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.791585    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7b65c1c2-a69a-444b-868e-2e8265cff8f3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.791917 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.791902    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.792390 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.792365    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.792518 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.792500    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7b65c1c2-a69a-444b-868e-2e8265cff8f3-config\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.797718 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.797700    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b27wc\" (UniqueName: \"kubernetes.io/projected/7b65c1c2-a69a-444b-868e-2e8265cff8f3-kube-api-access-b27wc\") pod \"prometheus-k8s-0\" (UID: \"7b65c1c2-a69a-444b-868e-2e8265cff8f3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:05.871743 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:05.871691    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:06.005967 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:06.005925    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 15 16:19:06.010597 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:19:06.010558    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b65c1c2_a69a_444b_868e_2e8265cff8f3.slice/crio-8071b35cba3ffba0f585a5dc539810ca0c5998cbdcfe97d55bda089e9cf9018d WatchSource:0}: Error finding container 8071b35cba3ffba0f585a5dc539810ca0c5998cbdcfe97d55bda089e9cf9018d: Status 404 returned error can't find the container with id 8071b35cba3ffba0f585a5dc539810ca0c5998cbdcfe97d55bda089e9cf9018d
Jul 15 16:19:06.503736 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:06.503703    2580 generic.go:356] "Generic (PLEG): container finished" podID="7b65c1c2-a69a-444b-868e-2e8265cff8f3" containerID="3f795201245b3f9527dbf7a95df31133ed43790fad75b46102dc29079f49395b" exitCode=0
Jul 15 16:19:06.503926 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:06.503786    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b65c1c2-a69a-444b-868e-2e8265cff8f3","Type":"ContainerDied","Data":"3f795201245b3f9527dbf7a95df31133ed43790fad75b46102dc29079f49395b"}
Jul 15 16:19:06.503926 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:06.503858    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b65c1c2-a69a-444b-868e-2e8265cff8f3","Type":"ContainerStarted","Data":"8071b35cba3ffba0f585a5dc539810ca0c5998cbdcfe97d55bda089e9cf9018d"}
Jul 15 16:19:06.877884 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:06.877856    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6" path="/var/lib/kubelet/pods/2f4a5e3c-b014-463a-96ff-3e2dbdbdf8c6/volumes"
Jul 15 16:19:07.513640 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:07.513603    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b65c1c2-a69a-444b-868e-2e8265cff8f3","Type":"ContainerStarted","Data":"69a487107fd13f6b3591352966eedca6c0a524f6e6db3bd8acc2f61a062864c6"}
Jul 15 16:19:07.514005 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:07.513675    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b65c1c2-a69a-444b-868e-2e8265cff8f3","Type":"ContainerStarted","Data":"625b4e68fd226b9534a9bdf71f31ce10b9304c1eabd71647255cfa40088a0ea9"}
Jul 15 16:19:07.514005 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:07.513702    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b65c1c2-a69a-444b-868e-2e8265cff8f3","Type":"ContainerStarted","Data":"30d7a08e71081b3f88b0f6cf4366050ddef0fe562b7ac6e68b2a21757e2a94ee"}
Jul 15 16:19:07.514005 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:07.513727    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b65c1c2-a69a-444b-868e-2e8265cff8f3","Type":"ContainerStarted","Data":"5bba5e15cbac053a73232399b418b92a865266253bca5d8862dac6e8e96454f6"}
Jul 15 16:19:07.514005 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:07.513749    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b65c1c2-a69a-444b-868e-2e8265cff8f3","Type":"ContainerStarted","Data":"26d6b3ffaf4cda1debf53482cdf63b9390d51f0ba0c5af3c129c47c7d414be60"}
Jul 15 16:19:07.514005 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:07.513792    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b65c1c2-a69a-444b-868e-2e8265cff8f3","Type":"ContainerStarted","Data":"a5346df4b0ba8fbacd5c6429c922f34be4b241fccf1b649b5330bafd7d0ce762"}
Jul 15 16:19:07.545680 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:07.545618    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.526149844 podStartE2EDuration="2.545598524s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.44868ms" isStatefulPod=false podCreationTimestamp="2026-07-15 16:19:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:19:07.544023892 +0000 UTC m=+149.191064428" watchObservedRunningTime="2026-07-15 16:19:07.545598524 +0000 UTC m=+149.192639060"
Jul 15 16:19:08.814082 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:08.814045    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:19:08.914833 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:08.914788    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:19:10.872282 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:10.872245    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:19:19.313134 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:19.313074    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:19:19.373288 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:19.373249    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:19:19.432751 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:19.432709    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:19:26.435140 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:26.435102    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:19:26.487494 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:19:26.487458    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:20:05.871905 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:20:05.871864    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:20:05.887290 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:20:05.887265    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:20:06.729996 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:20:06.729968    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 15 16:20:14.782427 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:20:14.782391    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:20:14.835981 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:20:14.835937    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:20:26.683939 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:20:26.683895    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:20:26.748435 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:20:26.748391    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:20:32.783365 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:20:32.783333    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:20:32.879830 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:20:32.879795    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:20:32.981589 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:20:32.981555    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:20:36.759573 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:20:36.759533    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:20:40.192748 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:20:40.192714    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:20:40.248924 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:20:40.248892    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:20:50.639015 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:20:50.638982    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:20:50.744127 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:20:50.744094    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:20:50.844453 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:20:50.844425    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:21:38.707491 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:21:38.707448    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-295d8_46356463-bfb1-42b8-8871-71357489a95b/ovn-acl-logging/0.log"
Jul 15 16:21:38.709598 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:21:38.709570    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-295d8_46356463-bfb1-42b8-8871-71357489a95b/ovn-acl-logging/0.log"
Jul 15 16:21:38.714327 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:21:38.714307    2580 kubelet.go:1648] "Image garbage collection succeeded"
Jul 15 16:21:45.505038 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:21:45.505008    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:21:52.241950 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:21:52.241913    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:21:54.787027 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:21:54.786990    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:21:54.945724 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:21:54.945690    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:22:14.021290 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:14.021256    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:22:26.908841 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:26.908797    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:22:27.984333 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:27.984300    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:22:30.806249 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:30.806171    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:22:30.872910 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:30.872878    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:22:32.244167 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:32.244130    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:22:32.363175 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:32.363141    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:22:32.481032 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:32.481003    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:22:34.705555 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:34.705519    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:22:34.912605 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:34.912574    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:22:42.312375 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:42.312337    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:22:44.643505 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:44.643465    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:22:44.743776 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:44.743734    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:22:46.007030 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:46.006995    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-57gsv"]
Jul 15 16:22:46.009245 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:46.009226    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-57gsv"
Jul 15 16:22:46.012063 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:46.012039    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 15 16:22:46.013030 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:46.012982    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-smvg2\""
Jul 15 16:22:46.013030 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:46.012993    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 15 16:22:46.018476 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:46.018445    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-57gsv"]
Jul 15 16:22:46.133312 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:46.133273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7k4x\" (UniqueName: \"kubernetes.io/projected/7156c58f-c0f1-4b88-a130-248479db8f72-kube-api-access-x7k4x\") pod \"openshift-lws-operator-fd8ccff4c-57gsv\" (UID: \"7156c58f-c0f1-4b88-a130-248479db8f72\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-57gsv"
Jul 15 16:22:46.133492 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:46.133322    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7156c58f-c0f1-4b88-a130-248479db8f72-tmp\") pod \"openshift-lws-operator-fd8ccff4c-57gsv\" (UID: \"7156c58f-c0f1-4b88-a130-248479db8f72\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-57gsv"
Jul 15 16:22:46.233778 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:46.233721    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7k4x\" (UniqueName: \"kubernetes.io/projected/7156c58f-c0f1-4b88-a130-248479db8f72-kube-api-access-x7k4x\") pod \"openshift-lws-operator-fd8ccff4c-57gsv\" (UID: \"7156c58f-c0f1-4b88-a130-248479db8f72\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-57gsv"
Jul 15 16:22:46.233956 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:46.233807    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7156c58f-c0f1-4b88-a130-248479db8f72-tmp\") pod \"openshift-lws-operator-fd8ccff4c-57gsv\" (UID: \"7156c58f-c0f1-4b88-a130-248479db8f72\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-57gsv"
Jul 15 16:22:46.234143 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:46.234124    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7156c58f-c0f1-4b88-a130-248479db8f72-tmp\") pod \"openshift-lws-operator-fd8ccff4c-57gsv\" (UID: \"7156c58f-c0f1-4b88-a130-248479db8f72\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-57gsv"
Jul 15 16:22:46.242802 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:46.242773    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7k4x\" (UniqueName: \"kubernetes.io/projected/7156c58f-c0f1-4b88-a130-248479db8f72-kube-api-access-x7k4x\") pod \"openshift-lws-operator-fd8ccff4c-57gsv\" (UID: \"7156c58f-c0f1-4b88-a130-248479db8f72\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-57gsv"
Jul 15 16:22:46.328359 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:46.328326    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-57gsv"
Jul 15 16:22:46.451390 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:46.451351    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-57gsv"]
Jul 15 16:22:46.454537 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:22:46.454509    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7156c58f_c0f1_4b88_a130_248479db8f72.slice/crio-5834f9d3bcebc828ea582b06b59cab9ab864510c4766daa7d6ef3b13cce994e1 WatchSource:0}: Error finding container 5834f9d3bcebc828ea582b06b59cab9ab864510c4766daa7d6ef3b13cce994e1: Status 404 returned error can't find the container with id 5834f9d3bcebc828ea582b06b59cab9ab864510c4766daa7d6ef3b13cce994e1
Jul 15 16:22:46.456015 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:46.455994    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 16:22:47.260656 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:47.260560    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-57gsv" event={"ID":"7156c58f-c0f1-4b88-a130-248479db8f72","Type":"ContainerStarted","Data":"5834f9d3bcebc828ea582b06b59cab9ab864510c4766daa7d6ef3b13cce994e1"}
Jul 15 16:22:49.269359 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:49.269257    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-57gsv" event={"ID":"7156c58f-c0f1-4b88-a130-248479db8f72","Type":"ContainerStarted","Data":"5e3179acddc5a32ebd4dd0c5c15f02fd2fc26bc08a8f7b38b36913c2ddc437b0"}
Jul 15 16:22:49.288574 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:49.288525    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-57gsv" podStartSLOduration=1.820950797 podStartE2EDuration="4.28850957s" totalImagesPullingTime="2.467558773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:22:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:22:49.286451254 +0000 UTC m=+370.933491792" watchObservedRunningTime="2026-07-15 16:22:49.28850957 +0000 UTC m=+370.935550105"
Jul 15 16:22:50.229221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:50.229191    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:22:50.729008 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:50.728974    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:22:52.459107 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:52.459070    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:22:54.459541 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:22:54.459510    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:06.189871 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:06.189832    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:06.282518 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:06.282481    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:06.893785 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:06.893741    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:07.203220 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.203129    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:07.324199 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.324166    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:07.381248 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.381214    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl"]
Jul 15 16:23:07.384531 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.384510    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl"
Jul 15 16:23:07.393091 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.393056    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 15 16:23:07.393284 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.393056    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 15 16:23:07.393284 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.393245    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-tc9mh\""
Jul 15 16:23:07.393427 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.393380    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 15 16:23:07.393549 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.393536    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 15 16:23:07.402448 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.402425    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fa739bac-a448-47d5-9af0-8fca1c693851-apiservice-cert\") pod \"opendatahub-operator-controller-manager-599cdff4cb-wxbdl\" (UID: \"fa739bac-a448-47d5-9af0-8fca1c693851\") " pod="opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl"
Jul 15 16:23:07.402542 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.402459    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa739bac-a448-47d5-9af0-8fca1c693851-tmp\") pod \"opendatahub-operator-controller-manager-599cdff4cb-wxbdl\" (UID: \"fa739bac-a448-47d5-9af0-8fca1c693851\") " pod="opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl"
Jul 15 16:23:07.402605 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.402547    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa739bac-a448-47d5-9af0-8fca1c693851-webhook-cert\") pod \"opendatahub-operator-controller-manager-599cdff4cb-wxbdl\" (UID: \"fa739bac-a448-47d5-9af0-8fca1c693851\") " pod="opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl"
Jul 15 16:23:07.402605 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.402594    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xphb\" (UniqueName: \"kubernetes.io/projected/fa739bac-a448-47d5-9af0-8fca1c693851-kube-api-access-2xphb\") pod \"opendatahub-operator-controller-manager-599cdff4cb-wxbdl\" (UID: \"fa739bac-a448-47d5-9af0-8fca1c693851\") " pod="opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl"
Jul 15 16:23:07.402704 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.402656    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fa739bac-a448-47d5-9af0-8fca1c693851-manifests\") pod \"opendatahub-operator-controller-manager-599cdff4cb-wxbdl\" (UID: \"fa739bac-a448-47d5-9af0-8fca1c693851\") " pod="opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl"
Jul 15 16:23:07.431068 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.431036    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl"]
Jul 15 16:23:07.503228 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.503126    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fa739bac-a448-47d5-9af0-8fca1c693851-apiservice-cert\") pod \"opendatahub-operator-controller-manager-599cdff4cb-wxbdl\" (UID: \"fa739bac-a448-47d5-9af0-8fca1c693851\") " pod="opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl"
Jul 15 16:23:07.503228 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.503200    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa739bac-a448-47d5-9af0-8fca1c693851-tmp\") pod \"opendatahub-operator-controller-manager-599cdff4cb-wxbdl\" (UID: \"fa739bac-a448-47d5-9af0-8fca1c693851\") " pod="opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl"
Jul 15 16:23:07.503445 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.503248    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa739bac-a448-47d5-9af0-8fca1c693851-webhook-cert\") pod \"opendatahub-operator-controller-manager-599cdff4cb-wxbdl\" (UID: \"fa739bac-a448-47d5-9af0-8fca1c693851\") " pod="opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl"
Jul 15 16:23:07.503445 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.503281    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xphb\" (UniqueName: \"kubernetes.io/projected/fa739bac-a448-47d5-9af0-8fca1c693851-kube-api-access-2xphb\") pod \"opendatahub-operator-controller-manager-599cdff4cb-wxbdl\" (UID: \"fa739bac-a448-47d5-9af0-8fca1c693851\") " pod="opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl"
Jul 15 16:23:07.503445 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.503315    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fa739bac-a448-47d5-9af0-8fca1c693851-manifests\") pod \"opendatahub-operator-controller-manager-599cdff4cb-wxbdl\" (UID: \"fa739bac-a448-47d5-9af0-8fca1c693851\") " pod="opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl"
Jul 15 16:23:07.503774 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.503735    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fa739bac-a448-47d5-9af0-8fca1c693851-manifests\") pod \"opendatahub-operator-controller-manager-599cdff4cb-wxbdl\" (UID: \"fa739bac-a448-47d5-9af0-8fca1c693851\") " pod="opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl"
Jul 15 16:23:07.505588 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.505563    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa739bac-a448-47d5-9af0-8fca1c693851-tmp\") pod \"opendatahub-operator-controller-manager-599cdff4cb-wxbdl\" (UID: \"fa739bac-a448-47d5-9af0-8fca1c693851\") " pod="opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl"
Jul 15 16:23:07.506106 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.506081    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa739bac-a448-47d5-9af0-8fca1c693851-webhook-cert\") pod \"opendatahub-operator-controller-manager-599cdff4cb-wxbdl\" (UID: \"fa739bac-a448-47d5-9af0-8fca1c693851\") " pod="opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl"
Jul 15 16:23:07.506224 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.506147    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fa739bac-a448-47d5-9af0-8fca1c693851-apiservice-cert\") pod \"opendatahub-operator-controller-manager-599cdff4cb-wxbdl\" (UID: \"fa739bac-a448-47d5-9af0-8fca1c693851\") " pod="opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl"
Jul 15 16:23:07.520673 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.520651    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xphb\" (UniqueName: \"kubernetes.io/projected/fa739bac-a448-47d5-9af0-8fca1c693851-kube-api-access-2xphb\") pod \"opendatahub-operator-controller-manager-599cdff4cb-wxbdl\" (UID: \"fa739bac-a448-47d5-9af0-8fca1c693851\") " pod="opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl"
Jul 15 16:23:07.633341 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.633303    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:07.694708 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.694675    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl"
Jul 15 16:23:07.742247 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.741602    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:07.833991 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.833968    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl"]
Jul 15 16:23:07.836048 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:23:07.836022    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa739bac_a448_47d5_9af0_8fca1c693851.slice/crio-4bbfdada0979e9a528faf6d20d8f42427af91dbb2415431efe6a7e82ce778c0e WatchSource:0}: Error finding container 4bbfdada0979e9a528faf6d20d8f42427af91dbb2415431efe6a7e82ce778c0e: Status 404 returned error can't find the container with id 4bbfdada0979e9a528faf6d20d8f42427af91dbb2415431efe6a7e82ce778c0e
Jul 15 16:23:07.837440 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.837404    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:07.947135 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:07.947101    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:08.126205 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:08.126176    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:08.229943 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:08.229899    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:08.338650 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:08.338616    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl" event={"ID":"fa739bac-a448-47d5-9af0-8fca1c693851","Type":"ContainerStarted","Data":"4bbfdada0979e9a528faf6d20d8f42427af91dbb2415431efe6a7e82ce778c0e"}
Jul 15 16:23:09.099976 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:09.099941    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:09.283844 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:09.283791    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:12.358295 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:12.358257    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl" event={"ID":"fa739bac-a448-47d5-9af0-8fca1c693851","Type":"ContainerStarted","Data":"33351474737dc49dd33c4d0f4e0a2a7b1b985b3777c46d259b38afea0317f26a"}
Jul 15 16:23:13.367938 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:13.367906    2580 generic.go:356] "Generic (PLEG): container finished" podID="fa739bac-a448-47d5-9af0-8fca1c693851" containerID="33351474737dc49dd33c4d0f4e0a2a7b1b985b3777c46d259b38afea0317f26a" exitCode=0
Jul 15 16:23:13.368334 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:13.367973    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl" event={"ID":"fa739bac-a448-47d5-9af0-8fca1c693851","Type":"ContainerDied","Data":"33351474737dc49dd33c4d0f4e0a2a7b1b985b3777c46d259b38afea0317f26a"}
Jul 15 16:23:14.373308 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:14.373262    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl" event={"ID":"fa739bac-a448-47d5-9af0-8fca1c693851","Type":"ContainerStarted","Data":"f7003201bc7aed6aa4b0e2fe1f755f5e9b247cff162c233ddd873ab94c5c0534"}
Jul 15 16:23:14.373690 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:14.373358    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl"
Jul 15 16:23:14.422158 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:14.422105    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl" podStartSLOduration=2.369498859 podStartE2EDuration="7.422088012s" totalImagesPullingTime="4.012061093s" totalInitContainerRuntime="1.04052806s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:23:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:23:14.420124039 +0000 UTC m=+396.067164576" watchObservedRunningTime="2026-07-15 16:23:14.422088012 +0000 UTC m=+396.069128547"
Jul 15 16:23:16.337770 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:16.337721    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:16.437960 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:16.437923    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:19.025011 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:19.024976    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:20.029962 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:20.029926    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:25.378726 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:25.378689    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-599cdff4cb-wxbdl"
Jul 15 16:23:27.719364 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:27.719332    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:27.923829 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:27.923797    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:28.128353 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:28.128319    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-84484457c8-wvxvj"]
Jul 15 16:23:28.139429 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:28.139403    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-84484457c8-wvxvj"
Jul 15 16:23:28.142216 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:28.142188    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 15 16:23:28.142463 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:28.142445    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 15 16:23:28.142940 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:28.142921    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 15 16:23:28.143032 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:28.142963    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 15 16:23:28.143913 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:28.143894    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:28.144630 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:28.144615    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-9vlxs\""
Jul 15 16:23:28.152882 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:28.152861    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-84484457c8-wvxvj"]
Jul 15 16:23:28.269104 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:28.269064    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99a21c19-71ef-400d-83a9-b2287f0f7c1f-tmp\") pod \"kube-auth-proxy-84484457c8-wvxvj\" (UID: \"99a21c19-71ef-400d-83a9-b2287f0f7c1f\") " pod="openshift-ingress/kube-auth-proxy-84484457c8-wvxvj"
Jul 15 16:23:28.269290 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:28.269131    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99a21c19-71ef-400d-83a9-b2287f0f7c1f-tls-certs\") pod \"kube-auth-proxy-84484457c8-wvxvj\" (UID: \"99a21c19-71ef-400d-83a9-b2287f0f7c1f\") " pod="openshift-ingress/kube-auth-proxy-84484457c8-wvxvj"
Jul 15 16:23:28.269290 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:28.269154    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9cfq\" (UniqueName: \"kubernetes.io/projected/99a21c19-71ef-400d-83a9-b2287f0f7c1f-kube-api-access-l9cfq\") pod \"kube-auth-proxy-84484457c8-wvxvj\" (UID: \"99a21c19-71ef-400d-83a9-b2287f0f7c1f\") " pod="openshift-ingress/kube-auth-proxy-84484457c8-wvxvj"
Jul 15 16:23:28.270000 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:28.269976    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:28.370478 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:28.370444    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99a21c19-71ef-400d-83a9-b2287f0f7c1f-tls-certs\") pod \"kube-auth-proxy-84484457c8-wvxvj\" (UID: \"99a21c19-71ef-400d-83a9-b2287f0f7c1f\") " pod="openshift-ingress/kube-auth-proxy-84484457c8-wvxvj"
Jul 15 16:23:28.370667 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:28.370483    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9cfq\" (UniqueName: \"kubernetes.io/projected/99a21c19-71ef-400d-83a9-b2287f0f7c1f-kube-api-access-l9cfq\") pod \"kube-auth-proxy-84484457c8-wvxvj\" (UID: \"99a21c19-71ef-400d-83a9-b2287f0f7c1f\") " pod="openshift-ingress/kube-auth-proxy-84484457c8-wvxvj"
Jul 15 16:23:28.370667 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:28.370524    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99a21c19-71ef-400d-83a9-b2287f0f7c1f-tmp\") pod \"kube-auth-proxy-84484457c8-wvxvj\" (UID: \"99a21c19-71ef-400d-83a9-b2287f0f7c1f\") " pod="openshift-ingress/kube-auth-proxy-84484457c8-wvxvj"
Jul 15 16:23:28.373010 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:28.372983    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99a21c19-71ef-400d-83a9-b2287f0f7c1f-tmp\") pod \"kube-auth-proxy-84484457c8-wvxvj\" (UID: \"99a21c19-71ef-400d-83a9-b2287f0f7c1f\") " pod="openshift-ingress/kube-auth-proxy-84484457c8-wvxvj"
Jul 15 16:23:28.373122 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:28.373091    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99a21c19-71ef-400d-83a9-b2287f0f7c1f-tls-certs\") pod \"kube-auth-proxy-84484457c8-wvxvj\" (UID: \"99a21c19-71ef-400d-83a9-b2287f0f7c1f\") " pod="openshift-ingress/kube-auth-proxy-84484457c8-wvxvj"
Jul 15 16:23:28.383481 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:28.383406    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9cfq\" (UniqueName: \"kubernetes.io/projected/99a21c19-71ef-400d-83a9-b2287f0f7c1f-kube-api-access-l9cfq\") pod \"kube-auth-proxy-84484457c8-wvxvj\" (UID: \"99a21c19-71ef-400d-83a9-b2287f0f7c1f\") " pod="openshift-ingress/kube-auth-proxy-84484457c8-wvxvj"
Jul 15 16:23:28.431488 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:28.431444    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:28.449306 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:28.449279    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-84484457c8-wvxvj"
Jul 15 16:23:28.570829 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:28.570795    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:28.607685 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:28.607647    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-84484457c8-wvxvj"]
Jul 15 16:23:28.610284 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:23:28.610252    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99a21c19_71ef_400d_83a9_b2287f0f7c1f.slice/crio-d41b7e13574ec3f8a33cbd53903f4673913ed50f3b70ac679bb85555b2170347 WatchSource:0}: Error finding container d41b7e13574ec3f8a33cbd53903f4673913ed50f3b70ac679bb85555b2170347: Status 404 returned error can't find the container with id d41b7e13574ec3f8a33cbd53903f4673913ed50f3b70ac679bb85555b2170347
Jul 15 16:23:28.744329 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:28.744233    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:29.363969 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.363935    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:29.427894 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.427807    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-84484457c8-wvxvj" event={"ID":"99a21c19-71ef-400d-83a9-b2287f0f7c1f","Type":"ContainerStarted","Data":"d41b7e13574ec3f8a33cbd53903f4673913ed50f3b70ac679bb85555b2170347"}
Jul 15 16:23:29.545643 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.545569    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:29.577364 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.577331    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-fh746"]
Jul 15 16:23:29.579705 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.579681    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:29.582605 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.582569    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 15 16:23:29.582742 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.582601    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-zgd6g\""
Jul 15 16:23:29.582742 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.582619    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 15 16:23:29.582742 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.582568    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 15 16:23:29.583197 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.583131    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 15 16:23:29.598506 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.598481    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-fh746"]
Jul 15 16:23:29.634322 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.634249    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:29.680699 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.680638    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/78802af0-77c0-44f3-9789-d69725d79adf-cacerts\") pod \"istiod-openshift-gateway-7574459656-fh746\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:29.680881 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.680767    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/78802af0-77c0-44f3-9789-d69725d79adf-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-fh746\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:29.680881 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.680814    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78802af0-77c0-44f3-9789-d69725d79adf-istio-token\") pod \"istiod-openshift-gateway-7574459656-fh746\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:29.680881 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.680857    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/78802af0-77c0-44f3-9789-d69725d79adf-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-fh746\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:29.681043 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.680920    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gks4d\" (UniqueName: \"kubernetes.io/projected/78802af0-77c0-44f3-9789-d69725d79adf-kube-api-access-gks4d\") pod \"istiod-openshift-gateway-7574459656-fh746\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:29.681043 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.680956    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/78802af0-77c0-44f3-9789-d69725d79adf-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-fh746\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:29.681120 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.681060    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/78802af0-77c0-44f3-9789-d69725d79adf-local-certs\") pod \"istiod-openshift-gateway-7574459656-fh746\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:29.781632 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.781593    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/78802af0-77c0-44f3-9789-d69725d79adf-local-certs\") pod \"istiod-openshift-gateway-7574459656-fh746\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:29.782112 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.781663    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/78802af0-77c0-44f3-9789-d69725d79adf-cacerts\") pod \"istiod-openshift-gateway-7574459656-fh746\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:29.782112 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.781712    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/78802af0-77c0-44f3-9789-d69725d79adf-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-fh746\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:29.782112 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.781742    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78802af0-77c0-44f3-9789-d69725d79adf-istio-token\") pod \"istiod-openshift-gateway-7574459656-fh746\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:29.782112 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.781786    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/78802af0-77c0-44f3-9789-d69725d79adf-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-fh746\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:29.782112 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.781821    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gks4d\" (UniqueName: \"kubernetes.io/projected/78802af0-77c0-44f3-9789-d69725d79adf-kube-api-access-gks4d\") pod \"istiod-openshift-gateway-7574459656-fh746\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:29.782112 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.781847    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/78802af0-77c0-44f3-9789-d69725d79adf-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-fh746\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:29.782618 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.782582    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/78802af0-77c0-44f3-9789-d69725d79adf-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-fh746\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:29.784335 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.784309    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/78802af0-77c0-44f3-9789-d69725d79adf-local-certs\") pod \"istiod-openshift-gateway-7574459656-fh746\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:29.784536 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.784515    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/78802af0-77c0-44f3-9789-d69725d79adf-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-fh746\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:29.784647 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.784622    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/78802af0-77c0-44f3-9789-d69725d79adf-cacerts\") pod \"istiod-openshift-gateway-7574459656-fh746\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:29.784743 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.784720    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/78802af0-77c0-44f3-9789-d69725d79adf-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-fh746\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:29.794572 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.794546    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78802af0-77c0-44f3-9789-d69725d79adf-istio-token\") pod \"istiod-openshift-gateway-7574459656-fh746\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:29.794720 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.794701    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gks4d\" (UniqueName: \"kubernetes.io/projected/78802af0-77c0-44f3-9789-d69725d79adf-kube-api-access-gks4d\") pod \"istiod-openshift-gateway-7574459656-fh746\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:29.891830 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:29.891729    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:30.034443 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:30.034418    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-fh746"]
Jul 15 16:23:30.036478 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:23:30.036450    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78802af0_77c0_44f3_9789_d69725d79adf.slice/crio-609dad39659cb1f9cf341f15d4c29c114dc25eb8d1ca8574a4c3c7b20c2d41d5 WatchSource:0}: Error finding container 609dad39659cb1f9cf341f15d4c29c114dc25eb8d1ca8574a4c3c7b20c2d41d5: Status 404 returned error can't find the container with id 609dad39659cb1f9cf341f15d4c29c114dc25eb8d1ca8574a4c3c7b20c2d41d5
Jul 15 16:23:30.432589 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:30.432553    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746" event={"ID":"78802af0-77c0-44f3-9789-d69725d79adf","Type":"ContainerStarted","Data":"609dad39659cb1f9cf341f15d4c29c114dc25eb8d1ca8574a4c3c7b20c2d41d5"}
Jul 15 16:23:30.823962 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:30.823921    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:32.982298 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:32.982262    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 15 16:23:32.982622 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:32.982330    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 15 16:23:33.448595 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:33.448554    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746" event={"ID":"78802af0-77c0-44f3-9789-d69725d79adf","Type":"ContainerStarted","Data":"a7ada55bf40a9ed5339c90066a41b580e67f5928f8914f91a8cd6dc7af112da9"}
Jul 15 16:23:33.448816 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:33.448619    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:33.450102 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:33.450076    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-84484457c8-wvxvj" event={"ID":"99a21c19-71ef-400d-83a9-b2287f0f7c1f","Type":"ContainerStarted","Data":"4f191fed2b734347eb6da89d8e8cacdb44932ee97983247260483eb55f987162"}
Jul 15 16:23:33.477966 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:33.477921    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746" podStartSLOduration=1.534319057 podStartE2EDuration="4.477906625s" totalImagesPullingTime="2.943587568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:23:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:23:33.477456357 +0000 UTC m=+415.124496894" watchObservedRunningTime="2026-07-15 16:23:33.477906625 +0000 UTC m=+415.124947160"
Jul 15 16:23:33.500745 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:33.500695    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-84484457c8-wvxvj" podStartSLOduration=2.045142331 podStartE2EDuration="5.500678885s" totalImagesPullingTime="3.455536554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:23:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:23:33.499666674 +0000 UTC m=+415.146707214" watchObservedRunningTime="2026-07-15 16:23:33.500678885 +0000 UTC m=+415.147719420"
Jul 15 16:23:34.090478 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:34.090447    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:34.238072 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:34.238034    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:34.456350 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:34.456263    2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-fh746 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 16:23:34.456350 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:34.456315    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746" podUID="78802af0-77c0-44f3-9789-d69725d79adf" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 16:23:34.537074 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:34.537038    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:35.160050 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:35.160016    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:35.461178 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:35.461134    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:35.632020 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:35.631988    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:37.455417 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.455389    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:37.666824 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.666788    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"]
Jul 15 16:23:37.670135 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.670112    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"
Jul 15 16:23:37.691311 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.691284    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"]
Jul 15 16:23:37.749328 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.749235    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-9t5cw\" (UID: \"6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"
Jul 15 16:23:37.749328 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.749269    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-9t5cw\" (UID: \"6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"
Jul 15 16:23:37.749328 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.749311    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-9t5cw\" (UID: \"6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"
Jul 15 16:23:37.749328 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.749330    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rbhc\" (UniqueName: \"kubernetes.io/projected/6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f-kube-api-access-7rbhc\") pod \"istiod-openshift-gateway-94bb8fbfd-9t5cw\" (UID: \"6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"
Jul 15 16:23:37.749625 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.749360    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-9t5cw\" (UID: \"6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"
Jul 15 16:23:37.749625 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.749385    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-9t5cw\" (UID: \"6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"
Jul 15 16:23:37.749625 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.749484    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-9t5cw\" (UID: \"6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"
Jul 15 16:23:37.850284 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.850245    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-9t5cw\" (UID: \"6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"
Jul 15 16:23:37.850284 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.850288    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rbhc\" (UniqueName: \"kubernetes.io/projected/6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f-kube-api-access-7rbhc\") pod \"istiod-openshift-gateway-94bb8fbfd-9t5cw\" (UID: \"6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"
Jul 15 16:23:37.850542 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.850319    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-9t5cw\" (UID: \"6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"
Jul 15 16:23:37.850542 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.850341    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-9t5cw\" (UID: \"6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"
Jul 15 16:23:37.850542 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.850362    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-9t5cw\" (UID: \"6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"
Jul 15 16:23:37.850542 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.850383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-9t5cw\" (UID: \"6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"
Jul 15 16:23:37.850542 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.850398    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-9t5cw\" (UID: \"6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"
Jul 15 16:23:37.851188 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.851157    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-9t5cw\" (UID: \"6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"
Jul 15 16:23:37.853080 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.853054    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-9t5cw\" (UID: \"6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"
Jul 15 16:23:37.853261 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.853242    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-9t5cw\" (UID: \"6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"
Jul 15 16:23:37.853330 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.853312    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-9t5cw\" (UID: \"6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"
Jul 15 16:23:37.853381 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.853357    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-9t5cw\" (UID: \"6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"
Jul 15 16:23:37.859561 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.859538    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rbhc\" (UniqueName: \"kubernetes.io/projected/6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f-kube-api-access-7rbhc\") pod \"istiod-openshift-gateway-94bb8fbfd-9t5cw\" (UID: \"6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"
Jul 15 16:23:37.859998 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.859977    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-9t5cw\" (UID: \"6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"
Jul 15 16:23:37.979073 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:37.979032    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"
Jul 15 16:23:38.156870 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:38.156836    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"]
Jul 15 16:23:38.159909 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:23:38.159884    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e8b2f22_aa9a_4b0c_93ea_0da4a88a631f.slice/crio-3b6c32afbb4bdd2e11705a01f2a91b30891f7aec0464fae6ce1294d4d6622b4c WatchSource:0}: Error finding container 3b6c32afbb4bdd2e11705a01f2a91b30891f7aec0464fae6ce1294d4d6622b4c: Status 404 returned error can't find the container with id 3b6c32afbb4bdd2e11705a01f2a91b30891f7aec0464fae6ce1294d4d6622b4c
Jul 15 16:23:38.162788 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:38.162746    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 15 16:23:38.162854 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:38.162817    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 15 16:23:38.472588 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:38.472548    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw" event={"ID":"6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f","Type":"ContainerStarted","Data":"f98ab4c1930bc86ac693a6b0c3d25ae41928247c4136a7a1c6f3149186bfb76a"}
Jul 15 16:23:38.473096 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:38.472612    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw" event={"ID":"6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f","Type":"ContainerStarted","Data":"3b6c32afbb4bdd2e11705a01f2a91b30891f7aec0464fae6ce1294d4d6622b4c"}
Jul 15 16:23:38.473096 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:38.472652    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"
Jul 15 16:23:38.498574 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:38.498518    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw" podStartSLOduration=1.4985017489999999 podStartE2EDuration="1.498501749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:23:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:23:38.497652916 +0000 UTC m=+420.144693473" watchObservedRunningTime="2026-07-15 16:23:38.498501749 +0000 UTC m=+420.145542275"
Jul 15 16:23:39.478215 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:39.478180    2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-9t5cw container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 16:23:39.478575 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:39.478246    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw" podUID="6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 16:23:39.555739 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:39.555708    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:39.667529 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:39.667498    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:39.862620 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:39.862589    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:40.157141 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:40.157054    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:40.854188 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:40.854155    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:40.961907 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:40.961871    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:41.057750 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:41.057718    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:42.477535 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:42.477495    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9t5cw"
Jul 15 16:23:42.539778 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:42.539722    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-fh746"]
Jul 15 16:23:42.539971 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:42.539935    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746" podUID="78802af0-77c0-44f3-9789-d69725d79adf" containerName="discovery" containerID="cri-o://a7ada55bf40a9ed5339c90066a41b580e67f5928f8914f91a8cd6dc7af112da9" gracePeriod=30
Jul 15 16:23:42.779222 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:42.779197    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:42.895843 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:42.895816    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78802af0-77c0-44f3-9789-d69725d79adf-istio-token\") pod \"78802af0-77c0-44f3-9789-d69725d79adf\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") "
Jul 15 16:23:42.896034 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:42.895848    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gks4d\" (UniqueName: \"kubernetes.io/projected/78802af0-77c0-44f3-9789-d69725d79adf-kube-api-access-gks4d\") pod \"78802af0-77c0-44f3-9789-d69725d79adf\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") "
Jul 15 16:23:42.896034 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:42.895900    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/78802af0-77c0-44f3-9789-d69725d79adf-istio-kubeconfig\") pod \"78802af0-77c0-44f3-9789-d69725d79adf\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") "
Jul 15 16:23:42.896034 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:42.895942    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/78802af0-77c0-44f3-9789-d69725d79adf-istio-csr-dns-cert\") pod \"78802af0-77c0-44f3-9789-d69725d79adf\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") "
Jul 15 16:23:42.896034 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:42.896008    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/78802af0-77c0-44f3-9789-d69725d79adf-istio-csr-ca-configmap\") pod \"78802af0-77c0-44f3-9789-d69725d79adf\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") "
Jul 15 16:23:42.896255 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:42.896082    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/78802af0-77c0-44f3-9789-d69725d79adf-cacerts\") pod \"78802af0-77c0-44f3-9789-d69725d79adf\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") "
Jul 15 16:23:42.896255 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:42.896108    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/78802af0-77c0-44f3-9789-d69725d79adf-local-certs\") pod \"78802af0-77c0-44f3-9789-d69725d79adf\" (UID: \"78802af0-77c0-44f3-9789-d69725d79adf\") "
Jul 15 16:23:42.896967 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:42.896866    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78802af0-77c0-44f3-9789-d69725d79adf-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "78802af0-77c0-44f3-9789-d69725d79adf" (UID: "78802af0-77c0-44f3-9789-d69725d79adf"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:23:42.898644 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:42.898593    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78802af0-77c0-44f3-9789-d69725d79adf-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "78802af0-77c0-44f3-9789-d69725d79adf" (UID: "78802af0-77c0-44f3-9789-d69725d79adf"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:23:42.898644 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:42.898624    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78802af0-77c0-44f3-9789-d69725d79adf-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "78802af0-77c0-44f3-9789-d69725d79adf" (UID: "78802af0-77c0-44f3-9789-d69725d79adf"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:23:42.898644 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:42.898633    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78802af0-77c0-44f3-9789-d69725d79adf-local-certs" (OuterVolumeSpecName: "local-certs") pod "78802af0-77c0-44f3-9789-d69725d79adf" (UID: "78802af0-77c0-44f3-9789-d69725d79adf"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:23:42.898861 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:42.898643    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78802af0-77c0-44f3-9789-d69725d79adf-kube-api-access-gks4d" (OuterVolumeSpecName: "kube-api-access-gks4d") pod "78802af0-77c0-44f3-9789-d69725d79adf" (UID: "78802af0-77c0-44f3-9789-d69725d79adf"). InnerVolumeSpecName "kube-api-access-gks4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:23:42.898861 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:42.898712    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78802af0-77c0-44f3-9789-d69725d79adf-istio-token" (OuterVolumeSpecName: "istio-token") pod "78802af0-77c0-44f3-9789-d69725d79adf" (UID: "78802af0-77c0-44f3-9789-d69725d79adf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:23:42.898861 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:42.898814    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78802af0-77c0-44f3-9789-d69725d79adf-cacerts" (OuterVolumeSpecName: "cacerts") pod "78802af0-77c0-44f3-9789-d69725d79adf" (UID: "78802af0-77c0-44f3-9789-d69725d79adf"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:23:42.996723 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:42.996677    2580 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/78802af0-77c0-44f3-9789-d69725d79adf-cacerts\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:23:42.996723 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:42.996707    2580 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/78802af0-77c0-44f3-9789-d69725d79adf-local-certs\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:23:42.996723 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:42.996721    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78802af0-77c0-44f3-9789-d69725d79adf-istio-token\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:23:42.996723 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:42.996733    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gks4d\" (UniqueName: \"kubernetes.io/projected/78802af0-77c0-44f3-9789-d69725d79adf-kube-api-access-gks4d\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:23:42.997083 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:42.996747    2580 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/78802af0-77c0-44f3-9789-d69725d79adf-istio-kubeconfig\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:23:42.997083 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:42.996783    2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/78802af0-77c0-44f3-9789-d69725d79adf-istio-csr-dns-cert\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:23:42.997083 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:42.996796    2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/78802af0-77c0-44f3-9789-d69725d79adf-istio-csr-ca-configmap\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:23:43.495483 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:43.495447    2580 generic.go:356] "Generic (PLEG): container finished" podID="78802af0-77c0-44f3-9789-d69725d79adf" containerID="a7ada55bf40a9ed5339c90066a41b580e67f5928f8914f91a8cd6dc7af112da9" exitCode=0
Jul 15 16:23:43.495933 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:43.495509    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746"
Jul 15 16:23:43.495933 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:43.495519    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746" event={"ID":"78802af0-77c0-44f3-9789-d69725d79adf","Type":"ContainerDied","Data":"a7ada55bf40a9ed5339c90066a41b580e67f5928f8914f91a8cd6dc7af112da9"}
Jul 15 16:23:43.495933 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:43.495565    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fh746" event={"ID":"78802af0-77c0-44f3-9789-d69725d79adf","Type":"ContainerDied","Data":"609dad39659cb1f9cf341f15d4c29c114dc25eb8d1ca8574a4c3c7b20c2d41d5"}
Jul 15 16:23:43.495933 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:43.495587    2580 scope.go:117] "RemoveContainer" containerID="a7ada55bf40a9ed5339c90066a41b580e67f5928f8914f91a8cd6dc7af112da9"
Jul 15 16:23:43.504043 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:43.504027    2580 scope.go:117] "RemoveContainer" containerID="a7ada55bf40a9ed5339c90066a41b580e67f5928f8914f91a8cd6dc7af112da9"
Jul 15 16:23:43.504300 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:23:43.504281    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7ada55bf40a9ed5339c90066a41b580e67f5928f8914f91a8cd6dc7af112da9\": container with ID starting with a7ada55bf40a9ed5339c90066a41b580e67f5928f8914f91a8cd6dc7af112da9 not found: ID does not exist" containerID="a7ada55bf40a9ed5339c90066a41b580e67f5928f8914f91a8cd6dc7af112da9"
Jul 15 16:23:43.504373 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:43.504310    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7ada55bf40a9ed5339c90066a41b580e67f5928f8914f91a8cd6dc7af112da9"} err="failed to get container status \"a7ada55bf40a9ed5339c90066a41b580e67f5928f8914f91a8cd6dc7af112da9\": rpc error: code = NotFound desc = could not find container \"a7ada55bf40a9ed5339c90066a41b580e67f5928f8914f91a8cd6dc7af112da9\": container with ID starting with a7ada55bf40a9ed5339c90066a41b580e67f5928f8914f91a8cd6dc7af112da9 not found: ID does not exist"
Jul 15 16:23:43.522300 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:43.522272    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-fh746"]
Jul 15 16:23:43.535446 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:43.535415    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-fh746"]
Jul 15 16:23:43.612147 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:43.612116    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:43.718050 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:43.718020    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:44.884215 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:44.884182    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78802af0-77c0-44f3-9789-d69725d79adf" path="/var/lib/kubelet/pods/78802af0-77c0-44f3-9789-d69725d79adf/volumes"
Jul 15 16:23:45.476464 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:45.476429    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:23:45.576858 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:23:45.576817    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:24:18.065133 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:18.065035    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:24:18.147442 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:18.147397    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:24:28.642325 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:28.642291    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:24:28.741170 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:28.741132    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:24:28.817506 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:28.817443    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:24:29.096866 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:29.096832    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:24:29.193859 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:29.193830    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:24:29.238079 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:29.238048    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b6lqw"]
Jul 15 16:24:29.238333 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:29.238321    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78802af0-77c0-44f3-9789-d69725d79adf" containerName="discovery"
Jul 15 16:24:29.238386 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:29.238335    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="78802af0-77c0-44f3-9789-d69725d79adf" containerName="discovery"
Jul 15 16:24:29.238386 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:29.238381    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="78802af0-77c0-44f3-9789-d69725d79adf" containerName="discovery"
Jul 15 16:24:29.243901 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:29.243876    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-b6lqw"
Jul 15 16:24:29.246706 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:29.246676    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 15 16:24:29.246706 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:29.246676    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 15 16:24:29.246898 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:29.246681    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-qnfjm\""
Jul 15 16:24:29.254858 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:29.254832    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b6lqw"]
Jul 15 16:24:29.302601 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:29.302556    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:24:29.393867 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:29.393751    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n6mf\" (UniqueName: \"kubernetes.io/projected/264d0a5b-1b97-4d53-b268-6cc6cafe53c8-kube-api-access-4n6mf\") pod \"kuadrant-operator-catalog-b6lqw\" (UID: \"264d0a5b-1b97-4d53-b268-6cc6cafe53c8\") " pod="kuadrant-system/kuadrant-operator-catalog-b6lqw"
Jul 15 16:24:29.494713 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:29.494657    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n6mf\" (UniqueName: \"kubernetes.io/projected/264d0a5b-1b97-4d53-b268-6cc6cafe53c8-kube-api-access-4n6mf\") pod \"kuadrant-operator-catalog-b6lqw\" (UID: \"264d0a5b-1b97-4d53-b268-6cc6cafe53c8\") " pod="kuadrant-system/kuadrant-operator-catalog-b6lqw"
Jul 15 16:24:29.517838 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:29.517802    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n6mf\" (UniqueName: \"kubernetes.io/projected/264d0a5b-1b97-4d53-b268-6cc6cafe53c8-kube-api-access-4n6mf\") pod \"kuadrant-operator-catalog-b6lqw\" (UID: \"264d0a5b-1b97-4d53-b268-6cc6cafe53c8\") " pod="kuadrant-system/kuadrant-operator-catalog-b6lqw"
Jul 15 16:24:29.554738 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:29.554698    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-b6lqw"
Jul 15 16:24:29.600226 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:29.600194    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b6lqw"]
Jul 15 16:24:29.683956 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:29.683921    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b6lqw"]
Jul 15 16:24:29.686107 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:24:29.686078    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod264d0a5b_1b97_4d53_b268_6cc6cafe53c8.slice/crio-4b6860db65dd59972a7a0af73b0474042034396a62368f7b3874e9789b974f31 WatchSource:0}: Error finding container 4b6860db65dd59972a7a0af73b0474042034396a62368f7b3874e9789b974f31: Status 404 returned error can't find the container with id 4b6860db65dd59972a7a0af73b0474042034396a62368f7b3874e9789b974f31
Jul 15 16:24:29.814224 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:29.814196    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7w6s2"]
Jul 15 16:24:29.817095 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:29.817080    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7w6s2"
Jul 15 16:24:29.826732 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:29.826712    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7w6s2"]
Jul 15 16:24:29.897310 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:29.897280    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftqrs\" (UniqueName: \"kubernetes.io/projected/a9ad7292-25ce-4a79-9601-daabe3cc9c7a-kube-api-access-ftqrs\") pod \"kuadrant-operator-catalog-7w6s2\" (UID: \"a9ad7292-25ce-4a79-9601-daabe3cc9c7a\") " pod="kuadrant-system/kuadrant-operator-catalog-7w6s2"
Jul 15 16:24:29.998808 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:29.998707    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftqrs\" (UniqueName: \"kubernetes.io/projected/a9ad7292-25ce-4a79-9601-daabe3cc9c7a-kube-api-access-ftqrs\") pod \"kuadrant-operator-catalog-7w6s2\" (UID: \"a9ad7292-25ce-4a79-9601-daabe3cc9c7a\") " pod="kuadrant-system/kuadrant-operator-catalog-7w6s2"
Jul 15 16:24:30.007206 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:30.007178    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftqrs\" (UniqueName: \"kubernetes.io/projected/a9ad7292-25ce-4a79-9601-daabe3cc9c7a-kube-api-access-ftqrs\") pod \"kuadrant-operator-catalog-7w6s2\" (UID: \"a9ad7292-25ce-4a79-9601-daabe3cc9c7a\") " pod="kuadrant-system/kuadrant-operator-catalog-7w6s2"
Jul 15 16:24:30.043992 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:30.043963    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:24:30.126392 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:30.126367    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7w6s2"
Jul 15 16:24:30.252669 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:30.252580    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:24:30.270585 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:30.270558    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7w6s2"]
Jul 15 16:24:30.297953 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:24:30.297915    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9ad7292_25ce_4a79_9601_daabe3cc9c7a.slice/crio-eaffd1dd19bac9e6fc2c566a6be31b4f1b1a87d8c44ccab6dd62b8e14e22cc99 WatchSource:0}: Error finding container eaffd1dd19bac9e6fc2c566a6be31b4f1b1a87d8c44ccab6dd62b8e14e22cc99: Status 404 returned error can't find the container with id eaffd1dd19bac9e6fc2c566a6be31b4f1b1a87d8c44ccab6dd62b8e14e22cc99
Jul 15 16:24:30.689137 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:30.689092    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-b6lqw" event={"ID":"264d0a5b-1b97-4d53-b268-6cc6cafe53c8","Type":"ContainerStarted","Data":"4b6860db65dd59972a7a0af73b0474042034396a62368f7b3874e9789b974f31"}
Jul 15 16:24:30.690130 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:30.690110    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7w6s2" event={"ID":"a9ad7292-25ce-4a79-9601-daabe3cc9c7a","Type":"ContainerStarted","Data":"eaffd1dd19bac9e6fc2c566a6be31b4f1b1a87d8c44ccab6dd62b8e14e22cc99"}
Jul 15 16:24:32.700388 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:32.700336    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-b6lqw" event={"ID":"264d0a5b-1b97-4d53-b268-6cc6cafe53c8","Type":"ContainerStarted","Data":"33b79b4d5189a5575905885f9daa60f77130fdb9d150e26854b17b891b1c38cc"}
Jul 15 16:24:32.700863 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:32.700428    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-b6lqw" podUID="264d0a5b-1b97-4d53-b268-6cc6cafe53c8" containerName="registry-server" containerID="cri-o://33b79b4d5189a5575905885f9daa60f77130fdb9d150e26854b17b891b1c38cc" gracePeriod=2
Jul 15 16:24:32.701907 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:32.701885    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7w6s2" event={"ID":"a9ad7292-25ce-4a79-9601-daabe3cc9c7a","Type":"ContainerStarted","Data":"304a4fa44f7cb7e38a6c74759a001f00d08f22d8b944c8de46476c556907a23c"}
Jul 15 16:24:32.717358 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:32.717316    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-b6lqw" podStartSLOduration=1.571413935 podStartE2EDuration="3.717302126s" totalImagesPullingTime="2.145888191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:24:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:24:32.715913799 +0000 UTC m=+474.362954336" watchObservedRunningTime="2026-07-15 16:24:32.717302126 +0000 UTC m=+474.364342661"
Jul 15 16:24:32.734221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:32.734180    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-7w6s2" podStartSLOduration=2.196831153 podStartE2EDuration="3.73416684s" totalImagesPullingTime="1.537335687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:24:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:24:32.732871401 +0000 UTC m=+474.379911937" watchObservedRunningTime="2026-07-15 16:24:32.73416684 +0000 UTC m=+474.381207376"
Jul 15 16:24:32.944316 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:32.944292    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-b6lqw"
Jul 15 16:24:33.128048 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:33.128012    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4n6mf\" (UniqueName: \"kubernetes.io/projected/264d0a5b-1b97-4d53-b268-6cc6cafe53c8-kube-api-access-4n6mf\") pod \"264d0a5b-1b97-4d53-b268-6cc6cafe53c8\" (UID: \"264d0a5b-1b97-4d53-b268-6cc6cafe53c8\") "
Jul 15 16:24:33.130191 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:33.130153    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/264d0a5b-1b97-4d53-b268-6cc6cafe53c8-kube-api-access-4n6mf" (OuterVolumeSpecName: "kube-api-access-4n6mf") pod "264d0a5b-1b97-4d53-b268-6cc6cafe53c8" (UID: "264d0a5b-1b97-4d53-b268-6cc6cafe53c8"). InnerVolumeSpecName "kube-api-access-4n6mf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:24:33.228903 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:33.228868    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4n6mf\" (UniqueName: \"kubernetes.io/projected/264d0a5b-1b97-4d53-b268-6cc6cafe53c8-kube-api-access-4n6mf\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:24:33.707457 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:33.707419    2580 generic.go:356] "Generic (PLEG): container finished" podID="264d0a5b-1b97-4d53-b268-6cc6cafe53c8" containerID="33b79b4d5189a5575905885f9daa60f77130fdb9d150e26854b17b891b1c38cc" exitCode=0
Jul 15 16:24:33.707877 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:33.707461    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-b6lqw"
Jul 15 16:24:33.707877 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:33.707504    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-b6lqw" event={"ID":"264d0a5b-1b97-4d53-b268-6cc6cafe53c8","Type":"ContainerDied","Data":"33b79b4d5189a5575905885f9daa60f77130fdb9d150e26854b17b891b1c38cc"}
Jul 15 16:24:33.707877 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:33.707553    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-b6lqw" event={"ID":"264d0a5b-1b97-4d53-b268-6cc6cafe53c8","Type":"ContainerDied","Data":"4b6860db65dd59972a7a0af73b0474042034396a62368f7b3874e9789b974f31"}
Jul 15 16:24:33.707877 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:33.707573    2580 scope.go:117] "RemoveContainer" containerID="33b79b4d5189a5575905885f9daa60f77130fdb9d150e26854b17b891b1c38cc"
Jul 15 16:24:33.716240 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:33.716220    2580 scope.go:117] "RemoveContainer" containerID="33b79b4d5189a5575905885f9daa60f77130fdb9d150e26854b17b891b1c38cc"
Jul 15 16:24:33.716461 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:24:33.716440    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33b79b4d5189a5575905885f9daa60f77130fdb9d150e26854b17b891b1c38cc\": container with ID starting with 33b79b4d5189a5575905885f9daa60f77130fdb9d150e26854b17b891b1c38cc not found: ID does not exist" containerID="33b79b4d5189a5575905885f9daa60f77130fdb9d150e26854b17b891b1c38cc"
Jul 15 16:24:33.716522 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:33.716474    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33b79b4d5189a5575905885f9daa60f77130fdb9d150e26854b17b891b1c38cc"} err="failed to get container status \"33b79b4d5189a5575905885f9daa60f77130fdb9d150e26854b17b891b1c38cc\": rpc error: code = NotFound desc = could not find container \"33b79b4d5189a5575905885f9daa60f77130fdb9d150e26854b17b891b1c38cc\": container with ID starting with 33b79b4d5189a5575905885f9daa60f77130fdb9d150e26854b17b891b1c38cc not found: ID does not exist"
Jul 15 16:24:33.728087 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:33.728064    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b6lqw"]
Jul 15 16:24:33.734383 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:33.734361    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b6lqw"]
Jul 15 16:24:34.877689 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:34.877653    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="264d0a5b-1b97-4d53-b268-6cc6cafe53c8" path="/var/lib/kubelet/pods/264d0a5b-1b97-4d53-b268-6cc6cafe53c8/volumes"
Jul 15 16:24:40.127036 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:40.126998    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-7w6s2"
Jul 15 16:24:40.127495 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:40.127142    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-7w6s2"
Jul 15 16:24:40.148592 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:40.148571    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-7w6s2"
Jul 15 16:24:40.757454 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:24:40.757424    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-7w6s2"
Jul 15 16:25:12.827241 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:12.827201    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:25:16.585916 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:16.585874    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:25:17.688239 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:17.688204    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd"]
Jul 15 16:25:17.688604 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:17.688494    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="264d0a5b-1b97-4d53-b268-6cc6cafe53c8" containerName="registry-server"
Jul 15 16:25:17.688604 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:17.688505    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="264d0a5b-1b97-4d53-b268-6cc6cafe53c8" containerName="registry-server"
Jul 15 16:25:17.688604 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:17.688550    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="264d0a5b-1b97-4d53-b268-6cc6cafe53c8" containerName="registry-server"
Jul 15 16:25:17.692046 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:17.692025    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd"
Jul 15 16:25:17.694895 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:17.694869    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-lbnzn\""
Jul 15 16:25:17.703145 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:17.703120    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd"]
Jul 15 16:25:17.790306 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:17.790262    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e81708d-89ca-444e-8d62-457a268a66dc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zqmdd\" (UID: \"7e81708d-89ca-444e-8d62-457a268a66dc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd"
Jul 15 16:25:17.790497 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:17.790396    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mlxp\" (UniqueName: \"kubernetes.io/projected/7e81708d-89ca-444e-8d62-457a268a66dc-kube-api-access-5mlxp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zqmdd\" (UID: \"7e81708d-89ca-444e-8d62-457a268a66dc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd"
Jul 15 16:25:17.891060 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:17.891022    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e81708d-89ca-444e-8d62-457a268a66dc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zqmdd\" (UID: \"7e81708d-89ca-444e-8d62-457a268a66dc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd"
Jul 15 16:25:17.891231 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:17.891094    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mlxp\" (UniqueName: \"kubernetes.io/projected/7e81708d-89ca-444e-8d62-457a268a66dc-kube-api-access-5mlxp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zqmdd\" (UID: \"7e81708d-89ca-444e-8d62-457a268a66dc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd"
Jul 15 16:25:17.891439 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:17.891414    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e81708d-89ca-444e-8d62-457a268a66dc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zqmdd\" (UID: \"7e81708d-89ca-444e-8d62-457a268a66dc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd"
Jul 15 16:25:17.903490 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:17.903463    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mlxp\" (UniqueName: \"kubernetes.io/projected/7e81708d-89ca-444e-8d62-457a268a66dc-kube-api-access-5mlxp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zqmdd\" (UID: \"7e81708d-89ca-444e-8d62-457a268a66dc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd"
Jul 15 16:25:18.001640 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:18.001559    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd"
Jul 15 16:25:18.147680 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:18.147658    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd"]
Jul 15 16:25:18.150306 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:25:18.150276    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e81708d_89ca_444e_8d62_457a268a66dc.slice/crio-90c4364aebda0254bca27c1ce4414742f4fce0d8ddb1466224a4086e77197ab2 WatchSource:0}: Error finding container 90c4364aebda0254bca27c1ce4414742f4fce0d8ddb1466224a4086e77197ab2: Status 404 returned error can't find the container with id 90c4364aebda0254bca27c1ce4414742f4fce0d8ddb1466224a4086e77197ab2
Jul 15 16:25:18.359019 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:18.358980    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:25:18.883633 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:18.883599    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd" event={"ID":"7e81708d-89ca-444e-8d62-457a268a66dc","Type":"ContainerStarted","Data":"90c4364aebda0254bca27c1ce4414742f4fce0d8ddb1466224a4086e77197ab2"}
Jul 15 16:25:19.586606 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:19.586567    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-jnvd9"]
Jul 15 16:25:19.589669 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:19.589653    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jnvd9"
Jul 15 16:25:19.592263 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:19.592236    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 15 16:25:19.592401 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:19.592267    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-njb9z\""
Jul 15 16:25:19.600923 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:19.600902    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-jnvd9"]
Jul 15 16:25:19.707085 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:19.707051    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch42h\" (UniqueName: \"kubernetes.io/projected/644bc2c5-b077-4139-9ffe-48348c359e91-kube-api-access-ch42h\") pod \"dns-operator-controller-manager-864dd78c4c-jnvd9\" (UID: \"644bc2c5-b077-4139-9ffe-48348c359e91\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jnvd9"
Jul 15 16:25:19.807873 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:19.807829    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch42h\" (UniqueName: \"kubernetes.io/projected/644bc2c5-b077-4139-9ffe-48348c359e91-kube-api-access-ch42h\") pod \"dns-operator-controller-manager-864dd78c4c-jnvd9\" (UID: \"644bc2c5-b077-4139-9ffe-48348c359e91\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jnvd9"
Jul 15 16:25:19.816483 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:19.816456    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch42h\" (UniqueName: \"kubernetes.io/projected/644bc2c5-b077-4139-9ffe-48348c359e91-kube-api-access-ch42h\") pod \"dns-operator-controller-manager-864dd78c4c-jnvd9\" (UID: \"644bc2c5-b077-4139-9ffe-48348c359e91\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jnvd9"
Jul 15 16:25:19.900589 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:19.900507    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jnvd9"
Jul 15 16:25:20.051199 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:20.051165    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-jnvd9"]
Jul 15 16:25:20.055201 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:25:20.055175    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod644bc2c5_b077_4139_9ffe_48348c359e91.slice/crio-cdebb80f0d034c3a1a5e00ade9d47bf93a789fe5e6e64c26fbb2eaff2d0ea0c9 WatchSource:0}: Error finding container cdebb80f0d034c3a1a5e00ade9d47bf93a789fe5e6e64c26fbb2eaff2d0ea0c9: Status 404 returned error can't find the container with id cdebb80f0d034c3a1a5e00ade9d47bf93a789fe5e6e64c26fbb2eaff2d0ea0c9
Jul 15 16:25:20.286642 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:20.285346    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:25:20.392093 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:20.392058    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:25:20.894832 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:20.894784    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jnvd9" event={"ID":"644bc2c5-b077-4139-9ffe-48348c359e91","Type":"ContainerStarted","Data":"cdebb80f0d034c3a1a5e00ade9d47bf93a789fe5e6e64c26fbb2eaff2d0ea0c9"}
Jul 15 16:25:22.602921 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:22.602898    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m994x"]
Jul 15 16:25:22.607399 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:22.607381    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m994x"
Jul 15 16:25:22.610808 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:22.610783    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-p92dz\""
Jul 15 16:25:22.627485 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:22.627461    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m994x"]
Jul 15 16:25:22.733602 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:22.733569    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hhz6\" (UniqueName: \"kubernetes.io/projected/74f23c7f-a006-4b7d-99d1-2c6044323ea0-kube-api-access-9hhz6\") pod \"limitador-operator-controller-manager-859d7b78bd-m994x\" (UID: \"74f23c7f-a006-4b7d-99d1-2c6044323ea0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m994x"
Jul 15 16:25:22.834962 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:22.834859    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hhz6\" (UniqueName: \"kubernetes.io/projected/74f23c7f-a006-4b7d-99d1-2c6044323ea0-kube-api-access-9hhz6\") pod \"limitador-operator-controller-manager-859d7b78bd-m994x\" (UID: \"74f23c7f-a006-4b7d-99d1-2c6044323ea0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m994x"
Jul 15 16:25:22.844447 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:22.844411    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hhz6\" (UniqueName: \"kubernetes.io/projected/74f23c7f-a006-4b7d-99d1-2c6044323ea0-kube-api-access-9hhz6\") pod \"limitador-operator-controller-manager-859d7b78bd-m994x\" (UID: \"74f23c7f-a006-4b7d-99d1-2c6044323ea0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m994x"
Jul 15 16:25:22.907698 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:22.907655    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jnvd9" event={"ID":"644bc2c5-b077-4139-9ffe-48348c359e91","Type":"ContainerStarted","Data":"0a7237441874f449c5f0b2abf2dddd4e99bf3b79866c9eae4092f71dd047364a"}
Jul 15 16:25:22.907893 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:22.907796    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jnvd9"
Jul 15 16:25:22.917245 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:22.917217    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m994x"
Jul 15 16:25:22.932572 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:22.932506    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jnvd9" podStartSLOduration=1.419784549 podStartE2EDuration="3.932484301s" totalImagesPullingTime="2.512699752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:25:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:25:22.931366359 +0000 UTC m=+524.578406905" watchObservedRunningTime="2026-07-15 16:25:22.932484301 +0000 UTC m=+524.579524838"
Jul 15 16:25:23.050960 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:23.050935    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m994x"]
Jul 15 16:25:23.053261 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:25:23.053229    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74f23c7f_a006_4b7d_99d1_2c6044323ea0.slice/crio-3c6ace229f043f1f9715e2915b3fbec241c5ce674ccd2ec8c6bf041b148de502 WatchSource:0}: Error finding container 3c6ace229f043f1f9715e2915b3fbec241c5ce674ccd2ec8c6bf041b148de502: Status 404 returned error can't find the container with id 3c6ace229f043f1f9715e2915b3fbec241c5ce674ccd2ec8c6bf041b148de502
Jul 15 16:25:23.207998 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:23.207912    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:25:23.310537 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:23.310501    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:25:23.557859 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:23.557825    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:25:23.913865 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:23.913778    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m994x" event={"ID":"74f23c7f-a006-4b7d-99d1-2c6044323ea0","Type":"ContainerStarted","Data":"3c6ace229f043f1f9715e2915b3fbec241c5ce674ccd2ec8c6bf041b148de502"}
Jul 15 16:25:24.921070 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:24.921032    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd" event={"ID":"7e81708d-89ca-444e-8d62-457a268a66dc","Type":"ContainerStarted","Data":"c2386a41c78b4db6d611ad58928d5039eee78e7f113418bd1c6c8c08ccae3435"}
Jul 15 16:25:24.921512 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:24.921163    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd"
Jul 15 16:25:24.967039 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:24.966973    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd" podStartSLOduration=1.400719893 podStartE2EDuration="7.966951958s" totalImagesPullingTime="6.566232065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:25:24.966614837 +0000 UTC m=+526.613655375" watchObservedRunningTime="2026-07-15 16:25:24.966951958 +0000 UTC m=+526.613992495"
Jul 15 16:25:25.004879 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:25.004845    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:25:25.113770 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:25.113676    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:25:25.960386 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:25.960352    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-kcxqf"]
Jul 15 16:25:25.963968 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:25.963950    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kcxqf"
Jul 15 16:25:25.966563 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:25.966535    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-sbpd5\""
Jul 15 16:25:25.973727 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:25.973684    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-kcxqf"]
Jul 15 16:25:26.064083 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:26.064046    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hthg4\" (UniqueName: \"kubernetes.io/projected/285b6be6-04b2-472c-bac0-c15063da586f-kube-api-access-hthg4\") pod \"authorino-operator-75cc8d5cb7-kcxqf\" (UID: \"285b6be6-04b2-472c-bac0-c15063da586f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-kcxqf"
Jul 15 16:25:26.165082 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:26.165030    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hthg4\" (UniqueName: \"kubernetes.io/projected/285b6be6-04b2-472c-bac0-c15063da586f-kube-api-access-hthg4\") pod \"authorino-operator-75cc8d5cb7-kcxqf\" (UID: \"285b6be6-04b2-472c-bac0-c15063da586f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-kcxqf"
Jul 15 16:25:26.173305 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:26.173284    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hthg4\" (UniqueName: \"kubernetes.io/projected/285b6be6-04b2-472c-bac0-c15063da586f-kube-api-access-hthg4\") pod \"authorino-operator-75cc8d5cb7-kcxqf\" (UID: \"285b6be6-04b2-472c-bac0-c15063da586f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-kcxqf"
Jul 15 16:25:26.277186 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:26.277156    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kcxqf"
Jul 15 16:25:26.429958 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:26.429904    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-kcxqf"]
Jul 15 16:25:26.432664 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:25:26.432633    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod285b6be6_04b2_472c_bac0_c15063da586f.slice/crio-9d4ba2ddbbcb63fce11c7fbe90dcc030f9f794be16d292ebf2d2fc6c05da202e WatchSource:0}: Error finding container 9d4ba2ddbbcb63fce11c7fbe90dcc030f9f794be16d292ebf2d2fc6c05da202e: Status 404 returned error can't find the container with id 9d4ba2ddbbcb63fce11c7fbe90dcc030f9f794be16d292ebf2d2fc6c05da202e
Jul 15 16:25:26.933017 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:26.932975    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kcxqf" event={"ID":"285b6be6-04b2-472c-bac0-c15063da586f","Type":"ContainerStarted","Data":"9d4ba2ddbbcb63fce11c7fbe90dcc030f9f794be16d292ebf2d2fc6c05da202e"}
Jul 15 16:25:26.934698 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:26.934668    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m994x" event={"ID":"74f23c7f-a006-4b7d-99d1-2c6044323ea0","Type":"ContainerStarted","Data":"8ceb399a1c2d218b512977a233abb34d316e55cc1ea7b22019b41a8de6928b53"}
Jul 15 16:25:26.934895 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:26.934876    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m994x"
Jul 15 16:25:26.954119 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:26.954059    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m994x" podStartSLOduration=1.8171415 podStartE2EDuration="4.954040982s" totalImagesPullingTime="3.136899482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:25:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:25:26.953696904 +0000 UTC m=+528.600737481" watchObservedRunningTime="2026-07-15 16:25:26.954040982 +0000 UTC m=+528.601081527"
Jul 15 16:25:29.457537 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:29.457492    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:25:29.560517 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:29.560480    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:25:29.951303 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:29.951266    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kcxqf" event={"ID":"285b6be6-04b2-472c-bac0-c15063da586f","Type":"ContainerStarted","Data":"4644f5444a8f6fb02d1d5341f98c0ce617c0b8ae6a829bf3f38de9a9e986761c"}
Jul 15 16:25:29.951491 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:29.951386    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kcxqf"
Jul 15 16:25:29.969352 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:29.969300    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kcxqf" podStartSLOduration=1.6575585 podStartE2EDuration="4.969282774s" totalImagesPullingTime="3.311724274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:25:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:25:29.968995956 +0000 UTC m=+531.616036494" watchObservedRunningTime="2026-07-15 16:25:29.969282774 +0000 UTC m=+531.616323315"
Jul 15 16:25:33.916943 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:33.916868    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jnvd9"
Jul 15 16:25:35.595226 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:35.595184    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:25:35.670054 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:35.670022    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:25:35.930205 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:35.930123    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd"
Jul 15 16:25:37.590462 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:37.590425    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2vts8"]
Jul 15 16:25:37.593612 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:37.593587    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2vts8"
Jul 15 16:25:37.608600 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:37.608575    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2vts8"]
Jul 15 16:25:37.668567 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:37.668526    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7552\" (UniqueName: \"kubernetes.io/projected/c3b52ad8-cf27-432c-a8c1-a80bc1472b92-kube-api-access-b7552\") pod \"kuadrant-operator-controller-manager-55467d9f7-2vts8\" (UID: \"c3b52ad8-cf27-432c-a8c1-a80bc1472b92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2vts8"
Jul 15 16:25:37.668743 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:37.668599    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3b52ad8-cf27-432c-a8c1-a80bc1472b92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-2vts8\" (UID: \"c3b52ad8-cf27-432c-a8c1-a80bc1472b92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2vts8"
Jul 15 16:25:37.769118 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:37.769080    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3b52ad8-cf27-432c-a8c1-a80bc1472b92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-2vts8\" (UID: \"c3b52ad8-cf27-432c-a8c1-a80bc1472b92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2vts8"
Jul 15 16:25:37.769291 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:37.769144    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7552\" (UniqueName: \"kubernetes.io/projected/c3b52ad8-cf27-432c-a8c1-a80bc1472b92-kube-api-access-b7552\") pod \"kuadrant-operator-controller-manager-55467d9f7-2vts8\" (UID: \"c3b52ad8-cf27-432c-a8c1-a80bc1472b92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2vts8"
Jul 15 16:25:37.769477 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:37.769449    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3b52ad8-cf27-432c-a8c1-a80bc1472b92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-2vts8\" (UID: \"c3b52ad8-cf27-432c-a8c1-a80bc1472b92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2vts8"
Jul 15 16:25:37.778925 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:37.778890    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7552\" (UniqueName: \"kubernetes.io/projected/c3b52ad8-cf27-432c-a8c1-a80bc1472b92-kube-api-access-b7552\") pod \"kuadrant-operator-controller-manager-55467d9f7-2vts8\" (UID: \"c3b52ad8-cf27-432c-a8c1-a80bc1472b92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2vts8"
Jul 15 16:25:37.905547 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:37.905459    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2vts8"
Jul 15 16:25:37.913357 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:37.913327    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2vts8"]
Jul 15 16:25:37.921684 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:37.921653    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2vts8"]
Jul 15 16:25:37.942131 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:37.942106    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m994x"
Jul 15 16:25:37.945172 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:37.945149    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5xlbz"]
Jul 15 16:25:37.947491 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:37.947472    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5xlbz"
Jul 15 16:25:37.965027 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:37.965003    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5xlbz"]
Jul 15 16:25:37.978334 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:37.978307    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd"]
Jul 15 16:25:37.978496 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:37.978476    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd" podUID="7e81708d-89ca-444e-8d62-457a268a66dc" containerName="manager" containerID="cri-o://c2386a41c78b4db6d611ad58928d5039eee78e7f113418bd1c6c8c08ccae3435" gracePeriod=2
Jul 15 16:25:37.990983 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:37.990956    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd"]
Jul 15 16:25:38.010915 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.010886    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m994x"]
Jul 15 16:25:38.011065 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.011045    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m994x" podUID="74f23c7f-a006-4b7d-99d1-2c6044323ea0" containerName="manager" containerID="cri-o://8ceb399a1c2d218b512977a233abb34d316e55cc1ea7b22019b41a8de6928b53" gracePeriod=2
Jul 15 16:25:38.027826 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.027796    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m994x"]
Jul 15 16:25:38.033139 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:25:38.033108    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zqmdd\" is forbidden: User \"system:node:ip-10-0-134-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-132.ec2.internal' and this object" podUID="7e81708d-89ca-444e-8d62-457a268a66dc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd"
Jul 15 16:25:38.050825 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.050793    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zxf6c"]
Jul 15 16:25:38.051266 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.051248    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74f23c7f-a006-4b7d-99d1-2c6044323ea0" containerName="manager"
Jul 15 16:25:38.051313 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.051271    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="74f23c7f-a006-4b7d-99d1-2c6044323ea0" containerName="manager"
Jul 15 16:25:38.051313 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.051295    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e81708d-89ca-444e-8d62-457a268a66dc" containerName="manager"
Jul 15 16:25:38.051313 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.051305    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e81708d-89ca-444e-8d62-457a268a66dc" containerName="manager"
Jul 15 16:25:38.051415 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.051383    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e81708d-89ca-444e-8d62-457a268a66dc" containerName="manager"
Jul 15 16:25:38.051415 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.051394    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="74f23c7f-a006-4b7d-99d1-2c6044323ea0" containerName="manager"
Jul 15 16:25:38.053739 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.053720    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zxf6c"
Jul 15 16:25:38.076799 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.070463    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zxf6c"]
Jul 15 16:25:38.077101 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.077056    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7twmj\" (UniqueName: \"kubernetes.io/projected/40ce4d07-7de9-4bbd-a197-e15867d4bd31-kube-api-access-7twmj\") pod \"kuadrant-operator-controller-manager-55467d9f7-5xlbz\" (UID: \"40ce4d07-7de9-4bbd-a197-e15867d4bd31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5xlbz"
Jul 15 16:25:38.077223 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.077116    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40ce4d07-7de9-4bbd-a197-e15867d4bd31-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-5xlbz\" (UID: \"40ce4d07-7de9-4bbd-a197-e15867d4bd31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5xlbz"
Jul 15 16:25:38.098824 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:25:38.098783    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-m994x\" is forbidden: User \"system:node:ip-10-0-134-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-132.ec2.internal' and this object" podUID="74f23c7f-a006-4b7d-99d1-2c6044323ea0" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m994x"
Jul 15 16:25:38.100976 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:25:38.100951    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zqmdd\" is forbidden: User \"system:node:ip-10-0-134-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-132.ec2.internal' and this object" podUID="7e81708d-89ca-444e-8d62-457a268a66dc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd"
Jul 15 16:25:38.178036 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.177996    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7twmj\" (UniqueName: \"kubernetes.io/projected/40ce4d07-7de9-4bbd-a197-e15867d4bd31-kube-api-access-7twmj\") pod \"kuadrant-operator-controller-manager-55467d9f7-5xlbz\" (UID: \"40ce4d07-7de9-4bbd-a197-e15867d4bd31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5xlbz"
Jul 15 16:25:38.178163 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.178056    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40ce4d07-7de9-4bbd-a197-e15867d4bd31-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-5xlbz\" (UID: \"40ce4d07-7de9-4bbd-a197-e15867d4bd31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5xlbz"
Jul 15 16:25:38.178163 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.178093    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz6xc\" (UniqueName: \"kubernetes.io/projected/5b3f1b15-3376-46d0-8a45-aec700393f71-kube-api-access-fz6xc\") pod \"limitador-operator-controller-manager-859d7b78bd-zxf6c\" (UID: \"5b3f1b15-3376-46d0-8a45-aec700393f71\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zxf6c"
Jul 15 16:25:38.178569 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.178539    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40ce4d07-7de9-4bbd-a197-e15867d4bd31-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-5xlbz\" (UID: \"40ce4d07-7de9-4bbd-a197-e15867d4bd31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5xlbz"
Jul 15 16:25:38.195225 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.195194    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7twmj\" (UniqueName: \"kubernetes.io/projected/40ce4d07-7de9-4bbd-a197-e15867d4bd31-kube-api-access-7twmj\") pod \"kuadrant-operator-controller-manager-55467d9f7-5xlbz\" (UID: \"40ce4d07-7de9-4bbd-a197-e15867d4bd31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5xlbz"
Jul 15 16:25:38.234934 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.234900    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd"
Jul 15 16:25:38.237860 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:25:38.237824    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-m994x\" is forbidden: User \"system:node:ip-10-0-134-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-132.ec2.internal' and this object" podUID="74f23c7f-a006-4b7d-99d1-2c6044323ea0" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m994x"
Jul 15 16:25:38.240865 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:25:38.240837    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zqmdd\" is forbidden: User \"system:node:ip-10-0-134-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-132.ec2.internal' and this object" podUID="7e81708d-89ca-444e-8d62-457a268a66dc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd"
Jul 15 16:25:38.241451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.241436    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m994x"
Jul 15 16:25:38.243689 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:25:38.243669    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-m994x\" is forbidden: User \"system:node:ip-10-0-134-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-132.ec2.internal' and this object" podUID="74f23c7f-a006-4b7d-99d1-2c6044323ea0" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m994x"
Jul 15 16:25:38.245875 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:25:38.245853    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zqmdd\" is forbidden: User \"system:node:ip-10-0-134-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-132.ec2.internal' and this object" podUID="7e81708d-89ca-444e-8d62-457a268a66dc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd"
Jul 15 16:25:38.257525 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.257507    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5xlbz"
Jul 15 16:25:38.279339 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.279316    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hhz6\" (UniqueName: \"kubernetes.io/projected/74f23c7f-a006-4b7d-99d1-2c6044323ea0-kube-api-access-9hhz6\") pod \"74f23c7f-a006-4b7d-99d1-2c6044323ea0\" (UID: \"74f23c7f-a006-4b7d-99d1-2c6044323ea0\") "
Jul 15 16:25:38.279467 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.279374    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e81708d-89ca-444e-8d62-457a268a66dc-extensions-socket-volume\") pod \"7e81708d-89ca-444e-8d62-457a268a66dc\" (UID: \"7e81708d-89ca-444e-8d62-457a268a66dc\") "
Jul 15 16:25:38.279467 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.279440    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mlxp\" (UniqueName: \"kubernetes.io/projected/7e81708d-89ca-444e-8d62-457a268a66dc-kube-api-access-5mlxp\") pod \"7e81708d-89ca-444e-8d62-457a268a66dc\" (UID: \"7e81708d-89ca-444e-8d62-457a268a66dc\") "
Jul 15 16:25:38.279709 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.279685    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fz6xc\" (UniqueName: \"kubernetes.io/projected/5b3f1b15-3376-46d0-8a45-aec700393f71-kube-api-access-fz6xc\") pod \"limitador-operator-controller-manager-859d7b78bd-zxf6c\" (UID: \"5b3f1b15-3376-46d0-8a45-aec700393f71\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zxf6c"
Jul 15 16:25:38.280020 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.279989    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e81708d-89ca-444e-8d62-457a268a66dc-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7e81708d-89ca-444e-8d62-457a268a66dc" (UID: "7e81708d-89ca-444e-8d62-457a268a66dc"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:25:38.281641 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.281612    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e81708d-89ca-444e-8d62-457a268a66dc-kube-api-access-5mlxp" (OuterVolumeSpecName: "kube-api-access-5mlxp") pod "7e81708d-89ca-444e-8d62-457a268a66dc" (UID: "7e81708d-89ca-444e-8d62-457a268a66dc"). InnerVolumeSpecName "kube-api-access-5mlxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:25:38.281731 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.281657    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74f23c7f-a006-4b7d-99d1-2c6044323ea0-kube-api-access-9hhz6" (OuterVolumeSpecName: "kube-api-access-9hhz6") pod "74f23c7f-a006-4b7d-99d1-2c6044323ea0" (UID: "74f23c7f-a006-4b7d-99d1-2c6044323ea0"). InnerVolumeSpecName "kube-api-access-9hhz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:25:38.289808 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.289789    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz6xc\" (UniqueName: \"kubernetes.io/projected/5b3f1b15-3376-46d0-8a45-aec700393f71-kube-api-access-fz6xc\") pod \"limitador-operator-controller-manager-859d7b78bd-zxf6c\" (UID: \"5b3f1b15-3376-46d0-8a45-aec700393f71\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zxf6c"
Jul 15 16:25:38.380453 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.380410    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mlxp\" (UniqueName: \"kubernetes.io/projected/7e81708d-89ca-444e-8d62-457a268a66dc-kube-api-access-5mlxp\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:25:38.380453 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.380441    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hhz6\" (UniqueName: \"kubernetes.io/projected/74f23c7f-a006-4b7d-99d1-2c6044323ea0-kube-api-access-9hhz6\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:25:38.380453 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.380454    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e81708d-89ca-444e-8d62-457a268a66dc-extensions-socket-volume\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:25:38.393929 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.393904    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5xlbz"]
Jul 15 16:25:38.395771 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:25:38.395722    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40ce4d07_7de9_4bbd_a197_e15867d4bd31.slice/crio-8e824563530f25f57cfd6d00ca888558623fbf8d35e8ee4570a6041db280aacc WatchSource:0}: Error finding container 8e824563530f25f57cfd6d00ca888558623fbf8d35e8ee4570a6041db280aacc: Status 404 returned error can't find the container with id 8e824563530f25f57cfd6d00ca888558623fbf8d35e8ee4570a6041db280aacc
Jul 15 16:25:38.401433 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.401414    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zxf6c"
Jul 15 16:25:38.539527 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.539499    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zxf6c"]
Jul 15 16:25:38.541625 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:25:38.541598    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b3f1b15_3376_46d0_8a45_aec700393f71.slice/crio-9de5f52ac7742be34dbb42ceedbe0806fb59a3ec11cd849524c12fd3f245b2cf WatchSource:0}: Error finding container 9de5f52ac7742be34dbb42ceedbe0806fb59a3ec11cd849524c12fd3f245b2cf: Status 404 returned error can't find the container with id 9de5f52ac7742be34dbb42ceedbe0806fb59a3ec11cd849524c12fd3f245b2cf
Jul 15 16:25:38.877010 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:25:38.876969    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-m994x\" is forbidden: User \"system:node:ip-10-0-134-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-132.ec2.internal' and this object" podUID="74f23c7f-a006-4b7d-99d1-2c6044323ea0" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m994x"
Jul 15 16:25:38.879089 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:25:38.879054    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zqmdd\" is forbidden: User \"system:node:ip-10-0-134-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-132.ec2.internal' and this object" podUID="7e81708d-89ca-444e-8d62-457a268a66dc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd"
Jul 15 16:25:38.879357 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.879335    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74f23c7f-a006-4b7d-99d1-2c6044323ea0" path="/var/lib/kubelet/pods/74f23c7f-a006-4b7d-99d1-2c6044323ea0/volumes"
Jul 15 16:25:38.879791 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.879770    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e81708d-89ca-444e-8d62-457a268a66dc" path="/var/lib/kubelet/pods/7e81708d-89ca-444e-8d62-457a268a66dc/volumes"
Jul 15 16:25:38.990805 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.990747    2580 generic.go:356] "Generic (PLEG): container finished" podID="7e81708d-89ca-444e-8d62-457a268a66dc" containerID="c2386a41c78b4db6d611ad58928d5039eee78e7f113418bd1c6c8c08ccae3435" exitCode=0
Jul 15 16:25:38.990934 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.990818    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zqmdd"
Jul 15 16:25:38.990934 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.990842    2580 scope.go:117] "RemoveContainer" containerID="c2386a41c78b4db6d611ad58928d5039eee78e7f113418bd1c6c8c08ccae3435"
Jul 15 16:25:38.992443 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.992419    2580 generic.go:356] "Generic (PLEG): container finished" podID="74f23c7f-a006-4b7d-99d1-2c6044323ea0" containerID="8ceb399a1c2d218b512977a233abb34d316e55cc1ea7b22019b41a8de6928b53" exitCode=0
Jul 15 16:25:38.992546 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.992476    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m994x"
Jul 15 16:25:38.994991 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.994963    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5xlbz" event={"ID":"40ce4d07-7de9-4bbd-a197-e15867d4bd31","Type":"ContainerStarted","Data":"2c3f70310f99e4cd96e633ceca2e94550b8197628e0d68bab78d8f40c996df89"}
Jul 15 16:25:38.995157 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.995049    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5xlbz" event={"ID":"40ce4d07-7de9-4bbd-a197-e15867d4bd31","Type":"ContainerStarted","Data":"8e824563530f25f57cfd6d00ca888558623fbf8d35e8ee4570a6041db280aacc"}
Jul 15 16:25:38.995157 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.995104    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5xlbz"
Jul 15 16:25:38.996627 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.996608    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zxf6c" event={"ID":"5b3f1b15-3376-46d0-8a45-aec700393f71","Type":"ContainerStarted","Data":"8e05ce0e4eb0155307fb66430fcc493dde32eedc2748dc226a361f7a30329193"}
Jul 15 16:25:38.996724 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.996643    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zxf6c" event={"ID":"5b3f1b15-3376-46d0-8a45-aec700393f71","Type":"ContainerStarted","Data":"9de5f52ac7742be34dbb42ceedbe0806fb59a3ec11cd849524c12fd3f245b2cf"}
Jul 15 16:25:38.996724 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:38.996662    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zxf6c"
Jul 15 16:25:39.000511 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:39.000495    2580 scope.go:117] "RemoveContainer" containerID="c2386a41c78b4db6d611ad58928d5039eee78e7f113418bd1c6c8c08ccae3435"
Jul 15 16:25:39.000748 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:25:39.000732    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2386a41c78b4db6d611ad58928d5039eee78e7f113418bd1c6c8c08ccae3435\": container with ID starting with c2386a41c78b4db6d611ad58928d5039eee78e7f113418bd1c6c8c08ccae3435 not found: ID does not exist" containerID="c2386a41c78b4db6d611ad58928d5039eee78e7f113418bd1c6c8c08ccae3435"
Jul 15 16:25:39.000803 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:39.000772    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2386a41c78b4db6d611ad58928d5039eee78e7f113418bd1c6c8c08ccae3435"} err="failed to get container status \"c2386a41c78b4db6d611ad58928d5039eee78e7f113418bd1c6c8c08ccae3435\": rpc error: code = NotFound desc = could not find container \"c2386a41c78b4db6d611ad58928d5039eee78e7f113418bd1c6c8c08ccae3435\": container with ID starting with c2386a41c78b4db6d611ad58928d5039eee78e7f113418bd1c6c8c08ccae3435 not found: ID does not exist"
Jul 15 16:25:39.000803 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:39.000796    2580 scope.go:117] "RemoveContainer" containerID="8ceb399a1c2d218b512977a233abb34d316e55cc1ea7b22019b41a8de6928b53"
Jul 15 16:25:39.009047 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:39.009020    2580 scope.go:117] "RemoveContainer" containerID="8ceb399a1c2d218b512977a233abb34d316e55cc1ea7b22019b41a8de6928b53"
Jul 15 16:25:39.009369 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:25:39.009337    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ceb399a1c2d218b512977a233abb34d316e55cc1ea7b22019b41a8de6928b53\": container with ID starting with 8ceb399a1c2d218b512977a233abb34d316e55cc1ea7b22019b41a8de6928b53 not found: ID does not exist" containerID="8ceb399a1c2d218b512977a233abb34d316e55cc1ea7b22019b41a8de6928b53"
Jul 15 16:25:39.009454 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:39.009376    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ceb399a1c2d218b512977a233abb34d316e55cc1ea7b22019b41a8de6928b53"} err="failed to get container status \"8ceb399a1c2d218b512977a233abb34d316e55cc1ea7b22019b41a8de6928b53\": rpc error: code = NotFound desc = could not find container \"8ceb399a1c2d218b512977a233abb34d316e55cc1ea7b22019b41a8de6928b53\": container with ID starting with 8ceb399a1c2d218b512977a233abb34d316e55cc1ea7b22019b41a8de6928b53 not found: ID does not exist"
Jul 15 16:25:39.016656 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:39.016616    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5xlbz" podStartSLOduration=2.016604293 podStartE2EDuration="2.016604293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:25:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:25:39.014963642 +0000 UTC m=+540.662004180" watchObservedRunningTime="2026-07-15 16:25:39.016604293 +0000 UTC m=+540.663644830"
Jul 15 16:25:39.032679 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:39.032635    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zxf6c" podStartSLOduration=1.032622086 podStartE2EDuration="1.032622086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:25:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:25:39.032014784 +0000 UTC m=+540.679055321" watchObservedRunningTime="2026-07-15 16:25:39.032622086 +0000 UTC m=+540.679662694"
Jul 15 16:25:40.111675 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:40.111638    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:25:40.354585 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:40.354533    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:25:40.450654 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:25:40.450616    2580 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-2vts8_kuadrant-system_c3b52ad8-cf27-432c-a8c1-a80bc1472b92_0(2f4e19fcdcb35e58e7aed3363b97776d33625b77362d5198567f8d900c16ecf3): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-2vts8 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}"
Jul 15 16:25:40.450740 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:25:40.450676    2580 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-2vts8_kuadrant-system_c3b52ad8-cf27-432c-a8c1-a80bc1472b92_0(2f4e19fcdcb35e58e7aed3363b97776d33625b77362d5198567f8d900c16ecf3): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-2vts8 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2vts8"
Jul 15 16:25:40.453082 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:40.453058    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:25:40.957563 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:40.957535    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kcxqf"
Jul 15 16:25:41.007011 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:41.006979    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2vts8"
Jul 15 16:25:41.014220 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:41.014197    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2vts8"
Jul 15 16:25:41.016547 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:25:41.016521    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-2vts8\" is forbidden: User \"system:node:ip-10-0-134-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-132.ec2.internal' and this object" podUID="c3b52ad8-cf27-432c-a8c1-a80bc1472b92" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2vts8"
Jul 15 16:25:41.099695 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:41.099662    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3b52ad8-cf27-432c-a8c1-a80bc1472b92-extensions-socket-volume\") pod \"c3b52ad8-cf27-432c-a8c1-a80bc1472b92\" (UID: \"c3b52ad8-cf27-432c-a8c1-a80bc1472b92\") "
Jul 15 16:25:41.099857 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:41.099707    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7552\" (UniqueName: \"kubernetes.io/projected/c3b52ad8-cf27-432c-a8c1-a80bc1472b92-kube-api-access-b7552\") pod \"c3b52ad8-cf27-432c-a8c1-a80bc1472b92\" (UID: \"c3b52ad8-cf27-432c-a8c1-a80bc1472b92\") "
Jul 15 16:25:41.099965 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:41.099933    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3b52ad8-cf27-432c-a8c1-a80bc1472b92-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c3b52ad8-cf27-432c-a8c1-a80bc1472b92" (UID: "c3b52ad8-cf27-432c-a8c1-a80bc1472b92"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:25:41.101914 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:41.101884    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3b52ad8-cf27-432c-a8c1-a80bc1472b92-kube-api-access-b7552" (OuterVolumeSpecName: "kube-api-access-b7552") pod "c3b52ad8-cf27-432c-a8c1-a80bc1472b92" (UID: "c3b52ad8-cf27-432c-a8c1-a80bc1472b92"). InnerVolumeSpecName "kube-api-access-b7552". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:25:41.201033 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:41.201004    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3b52ad8-cf27-432c-a8c1-a80bc1472b92-extensions-socket-volume\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:25:41.201033 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:41.201034    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7552\" (UniqueName: \"kubernetes.io/projected/c3b52ad8-cf27-432c-a8c1-a80bc1472b92-kube-api-access-b7552\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:25:42.010869 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:42.010832    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2vts8"
Jul 15 16:25:42.018552 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:25:42.018521    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-2vts8\" is forbidden: User \"system:node:ip-10-0-134-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-132.ec2.internal' and this object" podUID="c3b52ad8-cf27-432c-a8c1-a80bc1472b92" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2vts8"
Jul 15 16:25:42.021570 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:25:42.021547    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-2vts8\" is forbidden: User \"system:node:ip-10-0-134-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-132.ec2.internal' and this object" podUID="c3b52ad8-cf27-432c-a8c1-a80bc1472b92" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-2vts8"
Jul 15 16:25:42.879116 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:42.879084    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3b52ad8-cf27-432c-a8c1-a80bc1472b92" path="/var/lib/kubelet/pods/c3b52ad8-cf27-432c-a8c1-a80bc1472b92/volumes"
Jul 15 16:25:45.143808 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:45.143748    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:25:45.244050 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:45.243999    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:25:49.424793 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:49.424738    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:25:49.502965 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:49.502933    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:25:50.004963 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:50.004926    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5xlbz"
Jul 15 16:25:50.005164 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:50.005010    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zxf6c"
Jul 15 16:25:50.393887 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:50.393854    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6n7sp"]
Jul 15 16:25:50.398929 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:50.398905    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6n7sp"
Jul 15 16:25:50.418138 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:50.418106    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6n7sp"]
Jul 15 16:25:50.473256 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:50.473220    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f76d32f-591a-424f-af39-bdb60437e319-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6n7sp\" (UID: \"7f76d32f-591a-424f-af39-bdb60437e319\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6n7sp"
Jul 15 16:25:50.473598 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:50.473279    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqzgn\" (UniqueName: \"kubernetes.io/projected/7f76d32f-591a-424f-af39-bdb60437e319-kube-api-access-sqzgn\") pod \"kuadrant-operator-controller-manager-86878c769f-6n7sp\" (UID: \"7f76d32f-591a-424f-af39-bdb60437e319\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6n7sp"
Jul 15 16:25:50.574575 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:50.574531    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqzgn\" (UniqueName: \"kubernetes.io/projected/7f76d32f-591a-424f-af39-bdb60437e319-kube-api-access-sqzgn\") pod \"kuadrant-operator-controller-manager-86878c769f-6n7sp\" (UID: \"7f76d32f-591a-424f-af39-bdb60437e319\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6n7sp"
Jul 15 16:25:50.574793 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:50.574624    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f76d32f-591a-424f-af39-bdb60437e319-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6n7sp\" (UID: \"7f76d32f-591a-424f-af39-bdb60437e319\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6n7sp"
Jul 15 16:25:50.575034 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:50.575011    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f76d32f-591a-424f-af39-bdb60437e319-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6n7sp\" (UID: \"7f76d32f-591a-424f-af39-bdb60437e319\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6n7sp"
Jul 15 16:25:50.585164 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:50.585114    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqzgn\" (UniqueName: \"kubernetes.io/projected/7f76d32f-591a-424f-af39-bdb60437e319-kube-api-access-sqzgn\") pod \"kuadrant-operator-controller-manager-86878c769f-6n7sp\" (UID: \"7f76d32f-591a-424f-af39-bdb60437e319\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6n7sp"
Jul 15 16:25:50.709389 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:50.709350    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6n7sp"
Jul 15 16:25:50.846625 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:25:50.846593    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f76d32f_591a_424f_af39_bdb60437e319.slice/crio-1ea4e693d88e8ab7da497d80d767bf204400e65bcf1c8f52122255ebae86e820 WatchSource:0}: Error finding container 1ea4e693d88e8ab7da497d80d767bf204400e65bcf1c8f52122255ebae86e820: Status 404 returned error can't find the container with id 1ea4e693d88e8ab7da497d80d767bf204400e65bcf1c8f52122255ebae86e820
Jul 15 16:25:50.851525 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:50.851502    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6n7sp"]
Jul 15 16:25:51.055834 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:51.055796    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6n7sp" event={"ID":"7f76d32f-591a-424f-af39-bdb60437e319","Type":"ContainerStarted","Data":"fe716ddcdfac77338c065cfd7ba649de76f22d96fb6e894dcef34edaa90a3db0"}
Jul 15 16:25:51.056033 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:51.055859    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6n7sp" event={"ID":"7f76d32f-591a-424f-af39-bdb60437e319","Type":"ContainerStarted","Data":"1ea4e693d88e8ab7da497d80d767bf204400e65bcf1c8f52122255ebae86e820"}
Jul 15 16:25:51.056033 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:51.055888    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6n7sp"
Jul 15 16:25:51.101211 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:51.101091    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6n7sp" podStartSLOduration=1.101074696 podStartE2EDuration="1.101074696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:25:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:25:51.098272777 +0000 UTC m=+552.745313303" watchObservedRunningTime="2026-07-15 16:25:51.101074696 +0000 UTC m=+552.748115232"
Jul 15 16:25:56.541138 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:56.541079    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:25:56.641340 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:25:56.641289    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:26:00.749790 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:00.749726    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:26:00.849642 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:00.849608    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:26:02.062510 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:02.062480    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6n7sp"
Jul 15 16:26:02.114520 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:02.114486    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5xlbz"]
Jul 15 16:26:02.114733 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:02.114684    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5xlbz" podUID="40ce4d07-7de9-4bbd-a197-e15867d4bd31" containerName="manager" containerID="cri-o://2c3f70310f99e4cd96e633ceca2e94550b8197628e0d68bab78d8f40c996df89" gracePeriod=10
Jul 15 16:26:02.354973 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:02.354948    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5xlbz"
Jul 15 16:26:02.477270 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:02.477230    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7twmj\" (UniqueName: \"kubernetes.io/projected/40ce4d07-7de9-4bbd-a197-e15867d4bd31-kube-api-access-7twmj\") pod \"40ce4d07-7de9-4bbd-a197-e15867d4bd31\" (UID: \"40ce4d07-7de9-4bbd-a197-e15867d4bd31\") "
Jul 15 16:26:02.477451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:02.477326    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40ce4d07-7de9-4bbd-a197-e15867d4bd31-extensions-socket-volume\") pod \"40ce4d07-7de9-4bbd-a197-e15867d4bd31\" (UID: \"40ce4d07-7de9-4bbd-a197-e15867d4bd31\") "
Jul 15 16:26:02.477711 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:02.477688    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40ce4d07-7de9-4bbd-a197-e15867d4bd31-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "40ce4d07-7de9-4bbd-a197-e15867d4bd31" (UID: "40ce4d07-7de9-4bbd-a197-e15867d4bd31"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:26:02.479322 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:02.479284    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40ce4d07-7de9-4bbd-a197-e15867d4bd31-kube-api-access-7twmj" (OuterVolumeSpecName: "kube-api-access-7twmj") pod "40ce4d07-7de9-4bbd-a197-e15867d4bd31" (UID: "40ce4d07-7de9-4bbd-a197-e15867d4bd31"). InnerVolumeSpecName "kube-api-access-7twmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:26:02.577837 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:02.577796    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40ce4d07-7de9-4bbd-a197-e15867d4bd31-extensions-socket-volume\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:26:02.577837 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:02.577830    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7twmj\" (UniqueName: \"kubernetes.io/projected/40ce4d07-7de9-4bbd-a197-e15867d4bd31-kube-api-access-7twmj\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:26:03.112436 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:03.112393    2580 generic.go:356] "Generic (PLEG): container finished" podID="40ce4d07-7de9-4bbd-a197-e15867d4bd31" containerID="2c3f70310f99e4cd96e633ceca2e94550b8197628e0d68bab78d8f40c996df89" exitCode=0
Jul 15 16:26:03.112927 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:03.112463    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5xlbz"
Jul 15 16:26:03.112927 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:03.112478    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5xlbz" event={"ID":"40ce4d07-7de9-4bbd-a197-e15867d4bd31","Type":"ContainerDied","Data":"2c3f70310f99e4cd96e633ceca2e94550b8197628e0d68bab78d8f40c996df89"}
Jul 15 16:26:03.112927 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:03.112544    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5xlbz" event={"ID":"40ce4d07-7de9-4bbd-a197-e15867d4bd31","Type":"ContainerDied","Data":"8e824563530f25f57cfd6d00ca888558623fbf8d35e8ee4570a6041db280aacc"}
Jul 15 16:26:03.112927 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:03.112577    2580 scope.go:117] "RemoveContainer" containerID="2c3f70310f99e4cd96e633ceca2e94550b8197628e0d68bab78d8f40c996df89"
Jul 15 16:26:03.120804 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:03.120608    2580 scope.go:117] "RemoveContainer" containerID="2c3f70310f99e4cd96e633ceca2e94550b8197628e0d68bab78d8f40c996df89"
Jul 15 16:26:03.120965 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:26:03.120943    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c3f70310f99e4cd96e633ceca2e94550b8197628e0d68bab78d8f40c996df89\": container with ID starting with 2c3f70310f99e4cd96e633ceca2e94550b8197628e0d68bab78d8f40c996df89 not found: ID does not exist" containerID="2c3f70310f99e4cd96e633ceca2e94550b8197628e0d68bab78d8f40c996df89"
Jul 15 16:26:03.121017 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:03.120974    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c3f70310f99e4cd96e633ceca2e94550b8197628e0d68bab78d8f40c996df89"} err="failed to get container status \"2c3f70310f99e4cd96e633ceca2e94550b8197628e0d68bab78d8f40c996df89\": rpc error: code = NotFound desc = could not find container \"2c3f70310f99e4cd96e633ceca2e94550b8197628e0d68bab78d8f40c996df89\": container with ID starting with 2c3f70310f99e4cd96e633ceca2e94550b8197628e0d68bab78d8f40c996df89 not found: ID does not exist"
Jul 15 16:26:03.130425 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:03.130400    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5xlbz"]
Jul 15 16:26:03.136618 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:03.136562    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5xlbz"]
Jul 15 16:26:04.878591 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:04.878557    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40ce4d07-7de9-4bbd-a197-e15867d4bd31" path="/var/lib/kubelet/pods/40ce4d07-7de9-4bbd-a197-e15867d4bd31/volumes"
Jul 15 16:26:05.554087 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:05.554053    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:26:05.656058 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:05.656023    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:26:06.740121 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:06.740090    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:26:09.463314 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:09.463275    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:26:20.936793 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:20.936706    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:26:21.043134 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:21.043092    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:26:21.147096 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:21.147059    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:26:21.408532 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:21.406242    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:26:21.408532 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:21.406981    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40ce4d07-7de9-4bbd-a197-e15867d4bd31" containerName="manager"
Jul 15 16:26:21.408532 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:21.407003    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="40ce4d07-7de9-4bbd-a197-e15867d4bd31" containerName="manager"
Jul 15 16:26:21.408532 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:21.407132    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="40ce4d07-7de9-4bbd-a197-e15867d4bd31" containerName="manager"
Jul 15 16:26:21.410301 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:21.410276    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"
Jul 15 16:26:21.413080 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:21.413048    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 15 16:26:21.413238 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:21.413217    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5shnt\""
Jul 15 16:26:21.417042 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:21.416984    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:26:21.431118 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:21.431080    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:26:21.541814 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:21.541774    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/62bd54a0-d8bf-4314-ab69-849a178c7cf9-config-file\") pod \"limitador-limitador-54d6d988d5-mlqmb\" (UID: \"62bd54a0-d8bf-4314-ab69-849a178c7cf9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"
Jul 15 16:26:21.541814 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:21.541818    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqhcz\" (UniqueName: \"kubernetes.io/projected/62bd54a0-d8bf-4314-ab69-849a178c7cf9-kube-api-access-sqhcz\") pod \"limitador-limitador-54d6d988d5-mlqmb\" (UID: \"62bd54a0-d8bf-4314-ab69-849a178c7cf9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"
Jul 15 16:26:21.642935 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:21.642901    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/62bd54a0-d8bf-4314-ab69-849a178c7cf9-config-file\") pod \"limitador-limitador-54d6d988d5-mlqmb\" (UID: \"62bd54a0-d8bf-4314-ab69-849a178c7cf9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"
Jul 15 16:26:21.643103 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:21.642941    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqhcz\" (UniqueName: \"kubernetes.io/projected/62bd54a0-d8bf-4314-ab69-849a178c7cf9-kube-api-access-sqhcz\") pod \"limitador-limitador-54d6d988d5-mlqmb\" (UID: \"62bd54a0-d8bf-4314-ab69-849a178c7cf9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"
Jul 15 16:26:21.643510 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:21.643490    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/62bd54a0-d8bf-4314-ab69-849a178c7cf9-config-file\") pod \"limitador-limitador-54d6d988d5-mlqmb\" (UID: \"62bd54a0-d8bf-4314-ab69-849a178c7cf9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"
Jul 15 16:26:21.653604 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:21.653570    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqhcz\" (UniqueName: \"kubernetes.io/projected/62bd54a0-d8bf-4314-ab69-849a178c7cf9-kube-api-access-sqhcz\") pod \"limitador-limitador-54d6d988d5-mlqmb\" (UID: \"62bd54a0-d8bf-4314-ab69-849a178c7cf9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"
Jul 15 16:26:21.722295 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:21.722217    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"
Jul 15 16:26:21.781186 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:21.781151    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:26:21.857259 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:21.857224    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:26:21.860335 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:26:21.860301    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62bd54a0_d8bf_4314_ab69_849a178c7cf9.slice/crio-f3a0d070f2d2c807ec6db03d3d672f3d440947966b2acd038770b141562f489f WatchSource:0}: Error finding container f3a0d070f2d2c807ec6db03d3d672f3d440947966b2acd038770b141562f489f: Status 404 returned error can't find the container with id f3a0d070f2d2c807ec6db03d3d672f3d440947966b2acd038770b141562f489f
Jul 15 16:26:21.902679 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:21.902646    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:26:22.196858 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:22.196821    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-mlqmb" event={"ID":"62bd54a0-d8bf-4314-ab69-849a178c7cf9","Type":"ContainerStarted","Data":"f3a0d070f2d2c807ec6db03d3d672f3d440947966b2acd038770b141562f489f"}
Jul 15 16:26:27.228626 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:27.228589    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-mlqmb" event={"ID":"62bd54a0-d8bf-4314-ab69-849a178c7cf9","Type":"ContainerStarted","Data":"50c011c50b28d93658c20c86028c3e6ef953242e444d31f2c42ece4f60b74734"}
Jul 15 16:26:27.229038 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:27.228790    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"
Jul 15 16:26:27.249323 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:27.249273    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-mlqmb" podStartSLOduration=1.917100152 podStartE2EDuration="6.249256807s" totalImagesPullingTime="4.332156655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:26:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:26:27.247929295 +0000 UTC m=+588.894969858" watchObservedRunningTime="2026-07-15 16:26:27.249256807 +0000 UTC m=+588.896297360"
Jul 15 16:26:38.234267 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:38.234233    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"
Jul 15 16:26:38.736729 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:38.736705    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-295d8_46356463-bfb1-42b8-8871-71357489a95b/ovn-acl-logging/0.log"
Jul 15 16:26:38.736928 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:38.736711    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-295d8_46356463-bfb1-42b8-8871-71357489a95b/ovn-acl-logging/0.log"
Jul 15 16:26:43.655641 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:43.655603    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:26:43.800932 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:43.800896    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:26:44.276285 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:44.276252    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:26:56.301189 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:56.301152    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:26:57.621074 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:26:57.621041    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:27:03.918905 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:27:03.918818    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:27:04.022542 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:27:04.022509    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:27:04.124946 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:27:04.124909    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:27:07.419051 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:27:07.418998    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:27:07.562522 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:27:07.562483    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:27:14.020801 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:27:14.020767    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:27:14.121480 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:27:14.121447    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:27:20.180310 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:27:20.180278    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:27:37.320420 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:27:37.320380    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:27:37.437270 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:27:37.437194    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:27:39.530135 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:27:39.530094    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:27:39.624709 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:27:39.624665    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:27:39.997139 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:27:39.997102    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:27:40.375698 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:27:40.375665    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:27:40.822828 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:27:40.822800    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:27:40.994728 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:27:40.994691    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:27:41.062387 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:27:41.062354    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:27:41.387895 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:27:41.387863    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:27:50.310893 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:27:50.310836    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:27:50.410403 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:27:50.410372    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:28:02.545539 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:02.545482    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:28:02.993277 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:02.993224    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:28:05.722852 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.722813    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"]
Jul 15 16:28:05.726464 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.726444    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"
Jul 15 16:28:05.728927 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.728906    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 15 16:28:05.729731 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.729712    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-cb2r6\""
Jul 15 16:28:05.729862 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.729713    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 15 16:28:05.729862 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.729834    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 15 16:28:05.737568 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.737548    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"]
Jul 15 16:28:05.748834 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.748810    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:28:05.812309 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.812277    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwx8d\" (UniqueName: \"kubernetes.io/projected/7017539a-d855-4338-94b7-69efcbecee4c-kube-api-access-fwx8d\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tnpg4\" (UID: \"7017539a-d855-4338-94b7-69efcbecee4c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"
Jul 15 16:28:05.812309 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.812313    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7017539a-d855-4338-94b7-69efcbecee4c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tnpg4\" (UID: \"7017539a-d855-4338-94b7-69efcbecee4c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"
Jul 15 16:28:05.812502 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.812351    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7017539a-d855-4338-94b7-69efcbecee4c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tnpg4\" (UID: \"7017539a-d855-4338-94b7-69efcbecee4c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"
Jul 15 16:28:05.812502 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.812409    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7017539a-d855-4338-94b7-69efcbecee4c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tnpg4\" (UID: \"7017539a-d855-4338-94b7-69efcbecee4c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"
Jul 15 16:28:05.812502 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.812485    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7017539a-d855-4338-94b7-69efcbecee4c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tnpg4\" (UID: \"7017539a-d855-4338-94b7-69efcbecee4c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"
Jul 15 16:28:05.812603 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.812513    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7017539a-d855-4338-94b7-69efcbecee4c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tnpg4\" (UID: \"7017539a-d855-4338-94b7-69efcbecee4c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"
Jul 15 16:28:05.913193 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.913159    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwx8d\" (UniqueName: \"kubernetes.io/projected/7017539a-d855-4338-94b7-69efcbecee4c-kube-api-access-fwx8d\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tnpg4\" (UID: \"7017539a-d855-4338-94b7-69efcbecee4c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"
Jul 15 16:28:05.913193 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.913200    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7017539a-d855-4338-94b7-69efcbecee4c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tnpg4\" (UID: \"7017539a-d855-4338-94b7-69efcbecee4c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"
Jul 15 16:28:05.913447 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.913324    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7017539a-d855-4338-94b7-69efcbecee4c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tnpg4\" (UID: \"7017539a-d855-4338-94b7-69efcbecee4c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"
Jul 15 16:28:05.913447 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.913356    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7017539a-d855-4338-94b7-69efcbecee4c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tnpg4\" (UID: \"7017539a-d855-4338-94b7-69efcbecee4c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"
Jul 15 16:28:05.913447 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.913395    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7017539a-d855-4338-94b7-69efcbecee4c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tnpg4\" (UID: \"7017539a-d855-4338-94b7-69efcbecee4c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"
Jul 15 16:28:05.913447 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.913416    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7017539a-d855-4338-94b7-69efcbecee4c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tnpg4\" (UID: \"7017539a-d855-4338-94b7-69efcbecee4c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"
Jul 15 16:28:05.913834 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.913812    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7017539a-d855-4338-94b7-69efcbecee4c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tnpg4\" (UID: \"7017539a-d855-4338-94b7-69efcbecee4c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"
Jul 15 16:28:05.913932 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.913834    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7017539a-d855-4338-94b7-69efcbecee4c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tnpg4\" (UID: \"7017539a-d855-4338-94b7-69efcbecee4c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"
Jul 15 16:28:05.913932 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.913860    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7017539a-d855-4338-94b7-69efcbecee4c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tnpg4\" (UID: \"7017539a-d855-4338-94b7-69efcbecee4c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"
Jul 15 16:28:05.915397 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.915376    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7017539a-d855-4338-94b7-69efcbecee4c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tnpg4\" (UID: \"7017539a-d855-4338-94b7-69efcbecee4c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"
Jul 15 16:28:05.915659 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.915642    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7017539a-d855-4338-94b7-69efcbecee4c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tnpg4\" (UID: \"7017539a-d855-4338-94b7-69efcbecee4c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"
Jul 15 16:28:05.921072 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:05.921051    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwx8d\" (UniqueName: \"kubernetes.io/projected/7017539a-d855-4338-94b7-69efcbecee4c-kube-api-access-fwx8d\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tnpg4\" (UID: \"7017539a-d855-4338-94b7-69efcbecee4c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"
Jul 15 16:28:06.036925 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:06.036832    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"
Jul 15 16:28:06.167524 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:06.167499    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"]
Jul 15 16:28:06.170159 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:28:06.170119    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7017539a_d855_4338_94b7_69efcbecee4c.slice/crio-6dfe8afb4e8868d2db878e59e95aeb8fc988f1e6e1d63714b6920f732ca67908 WatchSource:0}: Error finding container 6dfe8afb4e8868d2db878e59e95aeb8fc988f1e6e1d63714b6920f732ca67908: Status 404 returned error can't find the container with id 6dfe8afb4e8868d2db878e59e95aeb8fc988f1e6e1d63714b6920f732ca67908
Jul 15 16:28:06.171665 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:06.171639    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 16:28:06.480564 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:06.480528    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:28:06.663910 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:06.663868    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4" event={"ID":"7017539a-d855-4338-94b7-69efcbecee4c","Type":"ContainerStarted","Data":"6dfe8afb4e8868d2db878e59e95aeb8fc988f1e6e1d63714b6920f732ca67908"}
Jul 15 16:28:07.671356 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:07.670677    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4" event={"ID":"7017539a-d855-4338-94b7-69efcbecee4c","Type":"ContainerStarted","Data":"e67407694c86265062cf540b9b221f735f9412aeb6f452c7ef52438bab825f49"}
Jul 15 16:28:07.694043 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:07.693981    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4" podStartSLOduration=1.556903557 podStartE2EDuration="2.693961133s" totalImagesPullingTime="1.137057576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:28:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:28:07.691854724 +0000 UTC m=+689.338895261" watchObservedRunningTime="2026-07-15 16:28:07.693961133 +0000 UTC m=+689.341001674"
Jul 15 16:28:16.037779 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:16.037718    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"
Jul 15 16:28:16.038225 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:16.037802    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"
Jul 15 16:28:16.050333 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:16.050305    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"
Jul 15 16:28:16.726890 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:16.726863    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tnpg4"
Jul 15 16:28:24.157632 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:24.157589    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:28:24.260614 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:24.260578    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:28:30.765532 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:30.765453    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:28:31.789702 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:31.789667    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:28:37.552736 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:37.552695    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:28:38.991456 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:38.991414    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:28:46.804250 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:46.804212    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:28:48.781829 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:48.781792    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:28:55.569205 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:55.569166    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:28:56.194519 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:28:56.194476    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:29:00.598108 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:29:00.598041    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:30:00.177118 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:30:00.177039    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735550-s5msg"]
Jul 15 16:30:00.180239 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:30:00.180221    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735550-s5msg"
Jul 15 16:30:00.183251 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:30:00.183232    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lsp65\""
Jul 15 16:30:00.189202 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:30:00.189181    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735550-s5msg"
Jul 15 16:30:00.196853 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:30:00.196834    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735550-s5msg"]
Jul 15 16:30:00.332820 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:30:00.332793    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735550-s5msg"]
Jul 15 16:30:00.334864 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:30:00.334835    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99028ae3_a363_49d2_b2c3_b4a73e1e378f.slice/crio-90b60c9bae3d9496b137297700b46376b010a80a7823c4fc6d2b61dcb9e5e1f6 WatchSource:0}: Error finding container 90b60c9bae3d9496b137297700b46376b010a80a7823c4fc6d2b61dcb9e5e1f6: Status 404 returned error can't find the container with id 90b60c9bae3d9496b137297700b46376b010a80a7823c4fc6d2b61dcb9e5e1f6
Jul 15 16:30:00.349407 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:30:00.349379    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:30:00.424208 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:30:00.424160    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:30:01.179519 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:30:01.179484    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735550-s5msg" event={"ID":"99028ae3-a363-49d2-b2c3-b4a73e1e378f","Type":"ContainerStarted","Data":"90b60c9bae3d9496b137297700b46376b010a80a7823c4fc6d2b61dcb9e5e1f6"}
Jul 15 16:30:01.717114 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:30:01.717077    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:30:01.905587 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:30:01.905543    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:30:04.196415 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:30:04.196327    2580 generic.go:356] "Generic (PLEG): container finished" podID="99028ae3-a363-49d2-b2c3-b4a73e1e378f" containerID="e89e0d4aa82ce40fa626539cea3ed60d9186d2f4f579f3ff37f5d21cd61a22ee" exitCode=0
Jul 15 16:30:04.196415 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:30:04.196380    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735550-s5msg" event={"ID":"99028ae3-a363-49d2-b2c3-b4a73e1e378f","Type":"ContainerDied","Data":"e89e0d4aa82ce40fa626539cea3ed60d9186d2f4f579f3ff37f5d21cd61a22ee"}
Jul 15 16:30:05.330629 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:30:05.330604    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735550-s5msg"
Jul 15 16:30:06.206478 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:30:06.206438    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735550-s5msg" event={"ID":"99028ae3-a363-49d2-b2c3-b4a73e1e378f","Type":"ContainerDied","Data":"90b60c9bae3d9496b137297700b46376b010a80a7823c4fc6d2b61dcb9e5e1f6"}
Jul 15 16:30:06.206478 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:30:06.206460    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735550-s5msg"
Jul 15 16:30:06.206809 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:30:06.206495    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="90b60c9bae3d9496b137297700b46376b010a80a7823c4fc6d2b61dcb9e5e1f6"
Jul 15 16:30:30.196152 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:30:30.196109    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:30:41.594114 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:30:41.594079    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:30:52.490042 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:30:52.490001    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:31:00.422310 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:31:00.422273    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:31:03.331722 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:31:03.331680    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:31:11.083542 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:31:11.083503    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:31:14.093088 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:31:14.093026    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:31:15.969870 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:31:15.969835    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:31:37.801356 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:31:37.801295    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:31:38.761248 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:31:38.761216    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-295d8_46356463-bfb1-42b8-8871-71357489a95b/ovn-acl-logging/0.log"
Jul 15 16:31:38.762833 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:31:38.762807    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-295d8_46356463-bfb1-42b8-8871-71357489a95b/ovn-acl-logging/0.log"
Jul 15 16:31:48.299853 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:31:48.299813    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:31:54.919047 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:31:54.919007    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:31:55.108301 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:31:55.108266    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:31:55.187336 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:31:55.187241    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:32:01.498685 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:32:01.498640    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:32:05.520142 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:32:05.520109    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:32:08.215540 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:32:08.215502    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:32:08.304820 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:32:08.304786    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:32:25.396628 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:32:25.396594    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:32:35.107423 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:32:35.107383    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:32:45.788634 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:32:45.788572    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:32:48.008948 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:32:48.008907    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:32:59.804090 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:32:59.804006    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:33:09.990183 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:33:09.990145    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:33:19.291155 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:33:19.291119    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:33:30.290967 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:33:30.290923    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:33:40.176991 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:33:40.176955    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:33:52.237604 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:33:52.237567    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:33:52.292791 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:33:52.292746    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:34:02.648218 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:34:02.648183    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:34:13.504151 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:34:13.504110    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:34:26.841207 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:34:26.841170    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:34:26.943077 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:34:26.943032    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:34:35.686062 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:34:35.685979    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:34:47.782789 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:34:47.782736    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:34:53.345567 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:34:53.345530    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:35:03.294933 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:35:03.294897    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:35:12.786961 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:35:12.786919    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:35:22.493326 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:35:22.493292    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:35:32.391252 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:35:32.391211    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:35:41.399774 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:35:41.399704    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:36:07.747047 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:36:07.746965    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6n7sp"]
Jul 15 16:36:07.747529 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:36:07.747172    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6n7sp" podUID="7f76d32f-591a-424f-af39-bdb60437e319" containerName="manager" containerID="cri-o://fe716ddcdfac77338c065cfd7ba649de76f22d96fb6e894dcef34edaa90a3db0" gracePeriod=10
Jul 15 16:36:10.834887 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:36:10.834848    2580 generic.go:356] "Generic (PLEG): container finished" podID="7f76d32f-591a-424f-af39-bdb60437e319" containerID="fe716ddcdfac77338c065cfd7ba649de76f22d96fb6e894dcef34edaa90a3db0" exitCode=0
Jul 15 16:36:10.835257 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:36:10.834907    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6n7sp" event={"ID":"7f76d32f-591a-424f-af39-bdb60437e319","Type":"ContainerDied","Data":"fe716ddcdfac77338c065cfd7ba649de76f22d96fb6e894dcef34edaa90a3db0"}
Jul 15 16:36:11.002548 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:36:11.002519    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6n7sp"
Jul 15 16:36:11.103911 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:36:11.103828    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f76d32f-591a-424f-af39-bdb60437e319-extensions-socket-volume\") pod \"7f76d32f-591a-424f-af39-bdb60437e319\" (UID: \"7f76d32f-591a-424f-af39-bdb60437e319\") "
Jul 15 16:36:11.104065 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:36:11.103924    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqzgn\" (UniqueName: \"kubernetes.io/projected/7f76d32f-591a-424f-af39-bdb60437e319-kube-api-access-sqzgn\") pod \"7f76d32f-591a-424f-af39-bdb60437e319\" (UID: \"7f76d32f-591a-424f-af39-bdb60437e319\") "
Jul 15 16:36:11.104264 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:36:11.104245    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f76d32f-591a-424f-af39-bdb60437e319-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7f76d32f-591a-424f-af39-bdb60437e319" (UID: "7f76d32f-591a-424f-af39-bdb60437e319"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:36:11.105980 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:36:11.105950    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f76d32f-591a-424f-af39-bdb60437e319-kube-api-access-sqzgn" (OuterVolumeSpecName: "kube-api-access-sqzgn") pod "7f76d32f-591a-424f-af39-bdb60437e319" (UID: "7f76d32f-591a-424f-af39-bdb60437e319"). InnerVolumeSpecName "kube-api-access-sqzgn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:36:11.205518 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:36:11.205478    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7f76d32f-591a-424f-af39-bdb60437e319-extensions-socket-volume\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:36:11.205518 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:36:11.205513    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqzgn\" (UniqueName: \"kubernetes.io/projected/7f76d32f-591a-424f-af39-bdb60437e319-kube-api-access-sqzgn\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:36:11.840821 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:36:11.840788    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6n7sp" event={"ID":"7f76d32f-591a-424f-af39-bdb60437e319","Type":"ContainerDied","Data":"1ea4e693d88e8ab7da497d80d767bf204400e65bcf1c8f52122255ebae86e820"}
Jul 15 16:36:11.841281 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:36:11.840838    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6n7sp"
Jul 15 16:36:11.841281 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:36:11.840849    2580 scope.go:117] "RemoveContainer" containerID="fe716ddcdfac77338c065cfd7ba649de76f22d96fb6e894dcef34edaa90a3db0"
Jul 15 16:36:11.865070 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:36:11.865041    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6n7sp"]
Jul 15 16:36:11.868958 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:36:11.868935    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6n7sp"]
Jul 15 16:36:12.879831 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:36:12.879798    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f76d32f-591a-424f-af39-bdb60437e319" path="/var/lib/kubelet/pods/7f76d32f-591a-424f-af39-bdb60437e319/volumes"
Jul 15 16:36:38.788498 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:36:38.788470    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-295d8_46356463-bfb1-42b8-8871-71357489a95b/ovn-acl-logging/0.log"
Jul 15 16:36:38.790885 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:36:38.790861    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-295d8_46356463-bfb1-42b8-8871-71357489a95b/ovn-acl-logging/0.log"
Jul 15 16:37:09.016467 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:09.016432    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhw65"]
Jul 15 16:37:09.018781 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:09.016744    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f76d32f-591a-424f-af39-bdb60437e319" containerName="manager"
Jul 15 16:37:09.018781 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:09.016767    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f76d32f-591a-424f-af39-bdb60437e319" containerName="manager"
Jul 15 16:37:09.018781 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:09.016784    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99028ae3-a363-49d2-b2c3-b4a73e1e378f" containerName="cleanup"
Jul 15 16:37:09.018781 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:09.016793    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="99028ae3-a363-49d2-b2c3-b4a73e1e378f" containerName="cleanup"
Jul 15 16:37:09.018781 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:09.016851    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f76d32f-591a-424f-af39-bdb60437e319" containerName="manager"
Jul 15 16:37:09.018781 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:09.016859    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="99028ae3-a363-49d2-b2c3-b4a73e1e378f" containerName="cleanup"
Jul 15 16:37:09.019668 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:09.019652    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhw65"
Jul 15 16:37:09.023003 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:09.022981    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-lbnzn\""
Jul 15 16:37:09.036102 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:09.036079    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhw65"]
Jul 15 16:37:09.191243 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:09.191195    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h848j\" (UniqueName: \"kubernetes.io/projected/34d0da2c-f49c-4747-87d1-d94807f4ce45-kube-api-access-h848j\") pod \"kuadrant-operator-controller-manager-86878c769f-xhw65\" (UID: \"34d0da2c-f49c-4747-87d1-d94807f4ce45\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhw65"
Jul 15 16:37:09.191427 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:09.191261    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34d0da2c-f49c-4747-87d1-d94807f4ce45-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xhw65\" (UID: \"34d0da2c-f49c-4747-87d1-d94807f4ce45\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhw65"
Jul 15 16:37:09.292128 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:09.292044    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h848j\" (UniqueName: \"kubernetes.io/projected/34d0da2c-f49c-4747-87d1-d94807f4ce45-kube-api-access-h848j\") pod \"kuadrant-operator-controller-manager-86878c769f-xhw65\" (UID: \"34d0da2c-f49c-4747-87d1-d94807f4ce45\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhw65"
Jul 15 16:37:09.292128 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:09.292095    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34d0da2c-f49c-4747-87d1-d94807f4ce45-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xhw65\" (UID: \"34d0da2c-f49c-4747-87d1-d94807f4ce45\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhw65"
Jul 15 16:37:09.292409 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:09.292391    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34d0da2c-f49c-4747-87d1-d94807f4ce45-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xhw65\" (UID: \"34d0da2c-f49c-4747-87d1-d94807f4ce45\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhw65"
Jul 15 16:37:09.306363 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:09.306330    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h848j\" (UniqueName: \"kubernetes.io/projected/34d0da2c-f49c-4747-87d1-d94807f4ce45-kube-api-access-h848j\") pod \"kuadrant-operator-controller-manager-86878c769f-xhw65\" (UID: \"34d0da2c-f49c-4747-87d1-d94807f4ce45\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhw65"
Jul 15 16:37:09.329878 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:09.329848    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhw65"
Jul 15 16:37:09.468721 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:09.468695    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhw65"]
Jul 15 16:37:09.471273 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:37:09.471240    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34d0da2c_f49c_4747_87d1_d94807f4ce45.slice/crio-3e050568c2ab16be2ac13fcad6831f57aefdda351a09170689e7e8d743c7637c WatchSource:0}: Error finding container 3e050568c2ab16be2ac13fcad6831f57aefdda351a09170689e7e8d743c7637c: Status 404 returned error can't find the container with id 3e050568c2ab16be2ac13fcad6831f57aefdda351a09170689e7e8d743c7637c
Jul 15 16:37:10.099015 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:10.098969    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhw65" event={"ID":"34d0da2c-f49c-4747-87d1-d94807f4ce45","Type":"ContainerStarted","Data":"90f6e2dff74becd66286689a54b9261f81fbb2cdb51ce07da52054d0f2874c7c"}
Jul 15 16:37:10.099492 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:10.099028    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhw65" event={"ID":"34d0da2c-f49c-4747-87d1-d94807f4ce45","Type":"ContainerStarted","Data":"3e050568c2ab16be2ac13fcad6831f57aefdda351a09170689e7e8d743c7637c"}
Jul 15 16:37:10.099492 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:10.099090    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhw65"
Jul 15 16:37:10.121173 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:10.121119    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhw65" podStartSLOduration=2.121100223 podStartE2EDuration="2.121100223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:37:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:37:10.118966726 +0000 UTC m=+1231.766007285" watchObservedRunningTime="2026-07-15 16:37:10.121100223 +0000 UTC m=+1231.768140740"
Jul 15 16:37:21.107291 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:21.107260    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xhw65"
Jul 15 16:37:29.884064 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:29.884022    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:37:39.087965 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:39.087929    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:37:51.786248 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:37:51.786202    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:38:01.983112 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:38:01.983064    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:38:12.383232 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:38:12.383187    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:38:23.003210 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:38:23.003172    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:38:33.592092 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:38:33.592046    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:38:43.388467 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:38:43.388424    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:38:55.207043 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:38:55.207002    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:39:03.883352 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:39:03.883266    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:39:14.885644 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:39:14.885603    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:39:25.661609 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:39:25.661572    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:39:36.384724 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:39:36.384686    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:39:46.277683 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:39:46.277647    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:39:55.383781 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:39:55.383735    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:40:07.886915 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:40:07.886880    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:40:16.979427 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:40:16.979389    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:40:25.285175 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:40:25.285140    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:40:34.382730 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:40:34.382637    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:40:39.729101 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:40:39.729065    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:40:43.874941 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:40:43.874910    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:40:55.685905 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:40:55.685870    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:41:04.586580 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:41:04.586538    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:41:14.186638 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:41:14.186596    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:41:27.784105 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:41:27.784064    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:41:36.985185 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:41:36.985146    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:41:38.816157 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:41:38.816123    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-295d8_46356463-bfb1-42b8-8871-71357489a95b/ovn-acl-logging/0.log"
Jul 15 16:41:38.820240 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:41:38.820212    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-295d8_46356463-bfb1-42b8-8871-71357489a95b/ovn-acl-logging/0.log"
Jul 15 16:41:46.786205 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:41:46.786172    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:41:59.685291 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:41:59.685254    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:42:10.939281 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:42:10.939246    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:42:21.806090 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:42:21.806053    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:42:22.897356 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:42:22.897316    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:42:31.392631 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:42:31.392592    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:42:36.318212 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:42:36.318166    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:42:42.186808 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:42:42.186742    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:42:47.490551 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:42:47.490506    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:42:59.894166 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:42:59.894131    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:43:00.384305 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:00.384263    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:43:07.976096 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:07.976058    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:43:10.984411 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:10.984376    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:43:16.265339 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:16.265298    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:43:21.420480 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:21.420439    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:43:24.725502 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:24.725456    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:43:24.825337 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:24.825298    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:43:25.030037 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:25.029952    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:43:25.202352 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:25.202309    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:43:31.556003 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:31.555893    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:43:31.637447 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:31.637413    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:43:31.854709 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:31.854607    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:43:32.018172 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:32.018138    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:43:33.014582 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:33.014550    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:43:33.111335 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:33.111301    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:43:33.253210 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:33.253169    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:43:33.514044 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:33.514007    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:43:33.807315 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:33.807278    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:43:33.914426 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:33.914383    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:43:35.887616 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:35.887572    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:43:36.050491 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:36.050460    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:43:38.119728 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:38.119695    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh"]
Jul 15 16:43:38.123548 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:38.123522    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh"
Jul 15 16:43:38.127377 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:38.127352    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e5f15b4-268d-4f82-92c7-4cb06c2005ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh\" (UID: \"8e5f15b4-268d-4f82-92c7-4cb06c2005ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh"
Jul 15 16:43:38.128572 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:38.128185    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lsp65\""
Jul 15 16:43:38.128572 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:38.128378    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 16:43:38.136206 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:38.136180    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh"]
Jul 15 16:43:38.228441 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:38.228405    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e5f15b4-268d-4f82-92c7-4cb06c2005ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh\" (UID: \"8e5f15b4-268d-4f82-92c7-4cb06c2005ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh"
Jul 15 16:43:38.229063 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:38.229041    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e5f15b4-268d-4f82-92c7-4cb06c2005ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh\" (UID: \"8e5f15b4-268d-4f82-92c7-4cb06c2005ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh"
Jul 15 16:43:38.436848 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:38.436740    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh"
Jul 15 16:43:38.566868 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:38.566839    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh"]
Jul 15 16:43:38.569416 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:43:38.569391    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e5f15b4_268d_4f82_92c7_4cb06c2005ff.slice/crio-03b3e1b8d458082333ebb23bfb41d520aafe27f2f18c9713f31f67b0d83759c8 WatchSource:0}: Error finding container 03b3e1b8d458082333ebb23bfb41d520aafe27f2f18c9713f31f67b0d83759c8: Status 404 returned error can't find the container with id 03b3e1b8d458082333ebb23bfb41d520aafe27f2f18c9713f31f67b0d83759c8
Jul 15 16:43:38.835829 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:38.835787    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh" event={"ID":"8e5f15b4-268d-4f82-92c7-4cb06c2005ff","Type":"ContainerStarted","Data":"0da70bb24058c81162c75c30eec958786d84aaa916cd49c1a23b7fc4ba9b569c"}
Jul 15 16:43:38.836013 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:38.835846    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh" event={"ID":"8e5f15b4-268d-4f82-92c7-4cb06c2005ff","Type":"ContainerStarted","Data":"03b3e1b8d458082333ebb23bfb41d520aafe27f2f18c9713f31f67b0d83759c8"}
Jul 15 16:43:38.852265 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:38.852213    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh" podStartSLOduration=0.852196931 podStartE2EDuration="852.196931ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:43:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:43:38.851490892 +0000 UTC m=+1620.498531449" watchObservedRunningTime="2026-07-15 16:43:38.852196931 +0000 UTC m=+1620.499237504"
Jul 15 16:43:39.842445 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:39.842404    2580 generic.go:356] "Generic (PLEG): container finished" podID="8e5f15b4-268d-4f82-92c7-4cb06c2005ff" containerID="0da70bb24058c81162c75c30eec958786d84aaa916cd49c1a23b7fc4ba9b569c" exitCode=7
Jul 15 16:43:39.842853 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:39.842470    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh" event={"ID":"8e5f15b4-268d-4f82-92c7-4cb06c2005ff","Type":"ContainerDied","Data":"0da70bb24058c81162c75c30eec958786d84aaa916cd49c1a23b7fc4ba9b569c"}
Jul 15 16:43:39.842853 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:39.842691    2580 scope.go:117] "RemoveContainer" containerID="0da70bb24058c81162c75c30eec958786d84aaa916cd49c1a23b7fc4ba9b569c"
Jul 15 16:43:40.850055 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:40.850021    2580 generic.go:356] "Generic (PLEG): container finished" podID="8e5f15b4-268d-4f82-92c7-4cb06c2005ff" containerID="90a27826e01b02cbd8370a8c7c15c0729a681ea737b734c4bcb0c0f7ac8d768e" exitCode=7
Jul 15 16:43:40.850549 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:40.850073    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh" event={"ID":"8e5f15b4-268d-4f82-92c7-4cb06c2005ff","Type":"ContainerDied","Data":"90a27826e01b02cbd8370a8c7c15c0729a681ea737b734c4bcb0c0f7ac8d768e"}
Jul 15 16:43:40.850549 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:40.850131    2580 scope.go:117] "RemoveContainer" containerID="0da70bb24058c81162c75c30eec958786d84aaa916cd49c1a23b7fc4ba9b569c"
Jul 15 16:43:40.850549 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:40.850312    2580 scope.go:117] "RemoveContainer" containerID="90a27826e01b02cbd8370a8c7c15c0729a681ea737b734c4bcb0c0f7ac8d768e"
Jul 15 16:43:40.850716 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:43:40.850569    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh_odh-ai-gateway-infra(8e5f15b4-268d-4f82-92c7-4cb06c2005ff)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh" podUID="8e5f15b4-268d-4f82-92c7-4cb06c2005ff"
Jul 15 16:43:41.857063 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:41.857040    2580 scope.go:117] "RemoveContainer" containerID="90a27826e01b02cbd8370a8c7c15c0729a681ea737b734c4bcb0c0f7ac8d768e"
Jul 15 16:43:41.857425 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:43:41.857222    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh_odh-ai-gateway-infra(8e5f15b4-268d-4f82-92c7-4cb06c2005ff)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh" podUID="8e5f15b4-268d-4f82-92c7-4cb06c2005ff"
Jul 15 16:43:44.758932 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:44.758893    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:43:44.860053 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:44.860019    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:43:52.874151 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:52.874119    2580 scope.go:117] "RemoveContainer" containerID="90a27826e01b02cbd8370a8c7c15c0729a681ea737b734c4bcb0c0f7ac8d768e"
Jul 15 16:43:53.913208 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:53.913176    2580 generic.go:356] "Generic (PLEG): container finished" podID="8e5f15b4-268d-4f82-92c7-4cb06c2005ff" containerID="902b6aaa1a549196b9e1a43351c26919fe737efea8ff8daa53bbb438e9244ed5" exitCode=0
Jul 15 16:43:53.913629 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:53.913220    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh" event={"ID":"8e5f15b4-268d-4f82-92c7-4cb06c2005ff","Type":"ContainerDied","Data":"902b6aaa1a549196b9e1a43351c26919fe737efea8ff8daa53bbb438e9244ed5"}
Jul 15 16:43:53.913629 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:53.913268    2580 scope.go:117] "RemoveContainer" containerID="90a27826e01b02cbd8370a8c7c15c0729a681ea737b734c4bcb0c0f7ac8d768e"
Jul 15 16:43:54.956847 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:54.956805    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh"]
Jul 15 16:43:55.054056 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:55.054027    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh"
Jul 15 16:43:55.155354 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:55.155320    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e5f15b4-268d-4f82-92c7-4cb06c2005ff-maas-api-service-ca\") pod \"8e5f15b4-268d-4f82-92c7-4cb06c2005ff\" (UID: \"8e5f15b4-268d-4f82-92c7-4cb06c2005ff\") "
Jul 15 16:43:55.155742 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:55.155712    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e5f15b4-268d-4f82-92c7-4cb06c2005ff-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8e5f15b4-268d-4f82-92c7-4cb06c2005ff" (UID: "8e5f15b4-268d-4f82-92c7-4cb06c2005ff"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:43:55.256571 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:55.256476    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e5f15b4-268d-4f82-92c7-4cb06c2005ff-maas-api-service-ca\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:43:55.925046 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:55.925015    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh"
Jul 15 16:43:55.925046 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:55.925032    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh" event={"ID":"8e5f15b4-268d-4f82-92c7-4cb06c2005ff","Type":"ContainerDied","Data":"03b3e1b8d458082333ebb23bfb41d520aafe27f2f18c9713f31f67b0d83759c8"}
Jul 15 16:43:55.925262 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:55.925076    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03b3e1b8d458082333ebb23bfb41d520aafe27f2f18c9713f31f67b0d83759c8"
Jul 15 16:43:55.955364 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:55.955327    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh"]
Jul 15 16:43:55.961328 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:55.961303    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-vxvwh"]
Jul 15 16:43:56.878663 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:43:56.878627    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e5f15b4-268d-4f82-92c7-4cb06c2005ff" path="/var/lib/kubelet/pods/8e5f15b4-268d-4f82-92c7-4cb06c2005ff/volumes"
Jul 15 16:44:28.167709 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:28.167666    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-dkbxf"]
Jul 15 16:44:28.168319 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:28.168182    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e5f15b4-268d-4f82-92c7-4cb06c2005ff" containerName="revoke-keys"
Jul 15 16:44:28.168319 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:28.168204    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e5f15b4-268d-4f82-92c7-4cb06c2005ff" containerName="revoke-keys"
Jul 15 16:44:28.168319 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:28.168223    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e5f15b4-268d-4f82-92c7-4cb06c2005ff" containerName="revoke-keys"
Jul 15 16:44:28.168319 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:28.168233    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e5f15b4-268d-4f82-92c7-4cb06c2005ff" containerName="revoke-keys"
Jul 15 16:44:28.168535 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:28.168322    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e5f15b4-268d-4f82-92c7-4cb06c2005ff" containerName="revoke-keys"
Jul 15 16:44:28.168535 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:28.168335    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e5f15b4-268d-4f82-92c7-4cb06c2005ff" containerName="revoke-keys"
Jul 15 16:44:28.168535 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:28.168346    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e5f15b4-268d-4f82-92c7-4cb06c2005ff" containerName="revoke-keys"
Jul 15 16:44:28.171343 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:28.171323    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-dkbxf"
Jul 15 16:44:28.173788 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:28.173767    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lsp65\""
Jul 15 16:44:28.174231 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:28.174217    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 16:44:28.179084 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:28.179064    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-dkbxf"]
Jul 15 16:44:28.258751 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:28.258712    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0fe66db-3795-4b1b-98ec-870f9466e9a9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-51d2510e-dkbxf\" (UID: \"d0fe66db-3795-4b1b-98ec-870f9466e9a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-dkbxf"
Jul 15 16:44:28.360172 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:28.360137    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0fe66db-3795-4b1b-98ec-870f9466e9a9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-51d2510e-dkbxf\" (UID: \"d0fe66db-3795-4b1b-98ec-870f9466e9a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-dkbxf"
Jul 15 16:44:28.360837 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:28.360811    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0fe66db-3795-4b1b-98ec-870f9466e9a9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-51d2510e-dkbxf\" (UID: \"d0fe66db-3795-4b1b-98ec-870f9466e9a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-dkbxf"
Jul 15 16:44:28.482905 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:28.482818    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-dkbxf"
Jul 15 16:44:28.610891 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:28.610868    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-dkbxf"]
Jul 15 16:44:28.613186 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:44:28.613158    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0fe66db_3795_4b1b_98ec_870f9466e9a9.slice/crio-fc0689feb5aeab91081d3e0f8a7069da0b8a86090a51349b4a909f99938b1f03 WatchSource:0}: Error finding container fc0689feb5aeab91081d3e0f8a7069da0b8a86090a51349b4a909f99938b1f03: Status 404 returned error can't find the container with id fc0689feb5aeab91081d3e0f8a7069da0b8a86090a51349b4a909f99938b1f03
Jul 15 16:44:29.076998 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:29.076958    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-dkbxf" event={"ID":"d0fe66db-3795-4b1b-98ec-870f9466e9a9","Type":"ContainerStarted","Data":"57824fa3a48f3488ac584b1000711f3919cebfa2c839e4774260dfc9c994dac2"}
Jul 15 16:44:29.077167 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:29.077013    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-dkbxf" event={"ID":"d0fe66db-3795-4b1b-98ec-870f9466e9a9","Type":"ContainerStarted","Data":"fc0689feb5aeab91081d3e0f8a7069da0b8a86090a51349b4a909f99938b1f03"}
Jul 15 16:44:29.096320 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:29.096192    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-dkbxf" podStartSLOduration=1.096175716 podStartE2EDuration="1.096175716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:44:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:44:29.094950603 +0000 UTC m=+1670.741991138" watchObservedRunningTime="2026-07-15 16:44:29.096175716 +0000 UTC m=+1670.743216251"
Jul 15 16:44:30.082837 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:30.082798    2580 generic.go:356] "Generic (PLEG): container finished" podID="d0fe66db-3795-4b1b-98ec-870f9466e9a9" containerID="57824fa3a48f3488ac584b1000711f3919cebfa2c839e4774260dfc9c994dac2" exitCode=0
Jul 15 16:44:30.083222 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:30.082883    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-dkbxf" event={"ID":"d0fe66db-3795-4b1b-98ec-870f9466e9a9","Type":"ContainerDied","Data":"57824fa3a48f3488ac584b1000711f3919cebfa2c839e4774260dfc9c994dac2"}
Jul 15 16:44:31.217091 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:31.217066    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-dkbxf"
Jul 15 16:44:31.281062 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:31.281024    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0fe66db-3795-4b1b-98ec-870f9466e9a9-maas-api-service-ca\") pod \"d0fe66db-3795-4b1b-98ec-870f9466e9a9\" (UID: \"d0fe66db-3795-4b1b-98ec-870f9466e9a9\") "
Jul 15 16:44:31.281377 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:31.281354    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0fe66db-3795-4b1b-98ec-870f9466e9a9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d0fe66db-3795-4b1b-98ec-870f9466e9a9" (UID: "d0fe66db-3795-4b1b-98ec-870f9466e9a9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:44:31.382064 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:31.381972    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0fe66db-3795-4b1b-98ec-870f9466e9a9-maas-api-service-ca\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:44:32.092389 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:32.092348    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-dkbxf" event={"ID":"d0fe66db-3795-4b1b-98ec-870f9466e9a9","Type":"ContainerDied","Data":"fc0689feb5aeab91081d3e0f8a7069da0b8a86090a51349b4a909f99938b1f03"}
Jul 15 16:44:32.092542 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:32.092408    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc0689feb5aeab91081d3e0f8a7069da0b8a86090a51349b4a909f99938b1f03"
Jul 15 16:44:32.092542 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:32.092361    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-51d2510e-dkbxf"
Jul 15 16:44:38.894422 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:38.894388    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:44:42.081741 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:42.081704    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:44:43.267280 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:43.267239    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:44:49.477471 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:49.477433    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:44:49.567527 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:49.567490    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:44:53.699183 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:53.699144    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:44:53.852141 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:53.852099    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:44:53.958463 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:53.958368    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:44:54.260668 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:54.260579    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:44:55.315115 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:55.315082    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:44:55.427627 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:55.427584    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:44:55.619379 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:55.619277    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:44:55.911249 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:55.911168    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:44:56.012173 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:56.012137    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:44:59.877930 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:59.877892    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr"]
Jul 15 16:44:59.878313 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:59.878255    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0fe66db-3795-4b1b-98ec-870f9466e9a9" containerName="revoke-keys"
Jul 15 16:44:59.878313 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:59.878267    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0fe66db-3795-4b1b-98ec-870f9466e9a9" containerName="revoke-keys"
Jul 15 16:44:59.878313 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:59.878278    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e5f15b4-268d-4f82-92c7-4cb06c2005ff" containerName="revoke-keys"
Jul 15 16:44:59.878313 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:59.878283    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e5f15b4-268d-4f82-92c7-4cb06c2005ff" containerName="revoke-keys"
Jul 15 16:44:59.878449 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:59.878346    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0fe66db-3795-4b1b-98ec-870f9466e9a9" containerName="revoke-keys"
Jul 15 16:44:59.881308 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:59.881292    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr"
Jul 15 16:44:59.884034 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:59.884004    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lsp65\""
Jul 15 16:44:59.884147 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:59.884010    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 16:44:59.889389 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:44:59.889357    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr"]
Jul 15 16:45:00.011675 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:00.011636    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/504b5c78-0f87-4a2b-bd61-2008d5650954-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr\" (UID: \"504b5c78-0f87-4a2b-bd61-2008d5650954\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr"
Jul 15 16:45:00.071163 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:00.071115    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:45:00.112601 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:00.112524    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/504b5c78-0f87-4a2b-bd61-2008d5650954-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr\" (UID: \"504b5c78-0f87-4a2b-bd61-2008d5650954\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr"
Jul 15 16:45:00.113177 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:00.113154    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/504b5c78-0f87-4a2b-bd61-2008d5650954-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr\" (UID: \"504b5c78-0f87-4a2b-bd61-2008d5650954\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr"
Jul 15 16:45:00.138638 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:00.138560    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-439e5009-29735565-2kqsv"]
Jul 15 16:45:00.142318 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:00.142293    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735565-4qmlx"]
Jul 15 16:45:00.142455 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:00.142400    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-439e5009-29735565-2kqsv"
Jul 15 16:45:00.145968 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:00.145944    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735565-4qmlx"
Jul 15 16:45:00.153257 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:00.153227    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-439e5009-29735565-2kqsv"]
Jul 15 16:45:00.155720 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:00.155692    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735565-4qmlx"]
Jul 15 16:45:00.156967 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:00.156940    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-439e5009-29735565-2kqsv"
Jul 15 16:45:00.165045 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:00.165021    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735565-4qmlx"
Jul 15 16:45:00.170619 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:00.170585    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:45:00.192661 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:00.192625    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr"
Jul 15 16:45:00.335660 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:00.335632    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735565-4qmlx"]
Jul 15 16:45:00.338808 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:45:00.338765    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eced033_1cb5_4264_9758_038529a62d22.slice/crio-bb01199120943735dea0e495e6913ad11f6ad77eddfbce30d17a911bdf46add9 WatchSource:0}: Error finding container bb01199120943735dea0e495e6913ad11f6ad77eddfbce30d17a911bdf46add9: Status 404 returned error can't find the container with id bb01199120943735dea0e495e6913ad11f6ad77eddfbce30d17a911bdf46add9
Jul 15 16:45:00.406803 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:00.406690    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr"]
Jul 15 16:45:00.411682 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:45:00.411648    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod504b5c78_0f87_4a2b_bd61_2008d5650954.slice/crio-9d875b86caf24e2102e2e3a84234f8479171850e8ba1d5f5410312e241e37c15 WatchSource:0}: Error finding container 9d875b86caf24e2102e2e3a84234f8479171850e8ba1d5f5410312e241e37c15: Status 404 returned error can't find the container with id 9d875b86caf24e2102e2e3a84234f8479171850e8ba1d5f5410312e241e37c15
Jul 15 16:45:00.557700 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:00.557667    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-439e5009-29735565-2kqsv"]
Jul 15 16:45:00.560544 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:45:00.560520    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9d33864_ccf0_4aee_9ac8_50e292417eaa.slice/crio-978f82739af7d2d80d75a951646f3d9c2ec09e11492c2a86ba58c19aeb5dd088 WatchSource:0}: Error finding container 978f82739af7d2d80d75a951646f3d9c2ec09e11492c2a86ba58c19aeb5dd088: Status 404 returned error can't find the container with id 978f82739af7d2d80d75a951646f3d9c2ec09e11492c2a86ba58c19aeb5dd088
Jul 15 16:45:01.229178 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:01.229077    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735565-4qmlx" event={"ID":"7eced033-1cb5-4264-9758-038529a62d22","Type":"ContainerStarted","Data":"b57f9c8f594c60f506557a2e14d38cb5695859f139f27c57174a6698af9effa5"}
Jul 15 16:45:01.229650 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:01.229180    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735565-4qmlx" event={"ID":"7eced033-1cb5-4264-9758-038529a62d22","Type":"ContainerStarted","Data":"bb01199120943735dea0e495e6913ad11f6ad77eddfbce30d17a911bdf46add9"}
Jul 15 16:45:01.230830 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:01.230800    2580 generic.go:356] "Generic (PLEG): container finished" podID="504b5c78-0f87-4a2b-bd61-2008d5650954" containerID="94e27f576864914987d50aae597669ffbd722315aeaf1509b4144326a6d11843" exitCode=7
Jul 15 16:45:01.230961 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:01.230885    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr" event={"ID":"504b5c78-0f87-4a2b-bd61-2008d5650954","Type":"ContainerDied","Data":"94e27f576864914987d50aae597669ffbd722315aeaf1509b4144326a6d11843"}
Jul 15 16:45:01.230961 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:01.230939    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr" event={"ID":"504b5c78-0f87-4a2b-bd61-2008d5650954","Type":"ContainerStarted","Data":"9d875b86caf24e2102e2e3a84234f8479171850e8ba1d5f5410312e241e37c15"}
Jul 15 16:45:01.231096 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:01.231082    2580 scope.go:117] "RemoveContainer" containerID="94e27f576864914987d50aae597669ffbd722315aeaf1509b4144326a6d11843"
Jul 15 16:45:01.232653 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:01.232633    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-439e5009-29735565-2kqsv" event={"ID":"a9d33864-ccf0-4aee-9ac8-50e292417eaa","Type":"ContainerStarted","Data":"06e67e13cc15d46637a9d80ca46449e8988dc5f0a6bc9550db6b2b83336de343"}
Jul 15 16:45:01.232781 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:01.232675    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-439e5009-29735565-2kqsv" event={"ID":"a9d33864-ccf0-4aee-9ac8-50e292417eaa","Type":"ContainerStarted","Data":"978f82739af7d2d80d75a951646f3d9c2ec09e11492c2a86ba58c19aeb5dd088"}
Jul 15 16:45:01.246672 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:01.246630    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735565-4qmlx" podStartSLOduration=1.246612458 podStartE2EDuration="1.246612458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:45:01.245387271 +0000 UTC m=+1702.892427807" watchObservedRunningTime="2026-07-15 16:45:01.246612458 +0000 UTC m=+1702.893652998"
Jul 15 16:45:01.262362 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:01.262305    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-439e5009-29735565-2kqsv" podStartSLOduration=1.262283917 podStartE2EDuration="1.262283917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:45:01.260663539 +0000 UTC m=+1702.907704099" watchObservedRunningTime="2026-07-15 16:45:01.262283917 +0000 UTC m=+1702.909324454"
Jul 15 16:45:02.239507 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:02.239426    2580 generic.go:356] "Generic (PLEG): container finished" podID="a9d33864-ccf0-4aee-9ac8-50e292417eaa" containerID="06e67e13cc15d46637a9d80ca46449e8988dc5f0a6bc9550db6b2b83336de343" exitCode=7
Jul 15 16:45:02.239888 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:02.239555    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-439e5009-29735565-2kqsv" event={"ID":"a9d33864-ccf0-4aee-9ac8-50e292417eaa","Type":"ContainerDied","Data":"06e67e13cc15d46637a9d80ca46449e8988dc5f0a6bc9550db6b2b83336de343"}
Jul 15 16:45:02.239888 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:02.239644    2580 scope.go:117] "RemoveContainer" containerID="06e67e13cc15d46637a9d80ca46449e8988dc5f0a6bc9550db6b2b83336de343"
Jul 15 16:45:02.241096 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:02.241076    2580 generic.go:356] "Generic (PLEG): container finished" podID="7eced033-1cb5-4264-9758-038529a62d22" containerID="b57f9c8f594c60f506557a2e14d38cb5695859f139f27c57174a6698af9effa5" exitCode=0
Jul 15 16:45:02.241186 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:02.241159    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735565-4qmlx" event={"ID":"7eced033-1cb5-4264-9758-038529a62d22","Type":"ContainerDied","Data":"b57f9c8f594c60f506557a2e14d38cb5695859f139f27c57174a6698af9effa5"}
Jul 15 16:45:02.242975 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:02.242927    2580 generic.go:356] "Generic (PLEG): container finished" podID="504b5c78-0f87-4a2b-bd61-2008d5650954" containerID="fd9e4c5a58bb383d3476ae36d041a37d23561a532898802558b8cb337c60ed4e" exitCode=7
Jul 15 16:45:02.242975 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:02.242965    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr" event={"ID":"504b5c78-0f87-4a2b-bd61-2008d5650954","Type":"ContainerDied","Data":"fd9e4c5a58bb383d3476ae36d041a37d23561a532898802558b8cb337c60ed4e"}
Jul 15 16:45:02.243095 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:02.243002    2580 scope.go:117] "RemoveContainer" containerID="94e27f576864914987d50aae597669ffbd722315aeaf1509b4144326a6d11843"
Jul 15 16:45:02.243209 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:02.243158    2580 scope.go:117] "RemoveContainer" containerID="fd9e4c5a58bb383d3476ae36d041a37d23561a532898802558b8cb337c60ed4e"
Jul 15 16:45:02.243357 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:45:02.243335    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr_odh-ai-gateway-infra(504b5c78-0f87-4a2b-bd61-2008d5650954)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr" podUID="504b5c78-0f87-4a2b-bd61-2008d5650954"
Jul 15 16:45:02.580528 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:02.580487    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:45:03.249577 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:03.249483    2580 scope.go:117] "RemoveContainer" containerID="fd9e4c5a58bb383d3476ae36d041a37d23561a532898802558b8cb337c60ed4e"
Jul 15 16:45:03.249976 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:45:03.249688    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr_odh-ai-gateway-infra(504b5c78-0f87-4a2b-bd61-2008d5650954)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr" podUID="504b5c78-0f87-4a2b-bd61-2008d5650954"
Jul 15 16:45:03.251023 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:03.250997    2580 generic.go:356] "Generic (PLEG): container finished" podID="a9d33864-ccf0-4aee-9ac8-50e292417eaa" containerID="f4d043040b749d61ce81e9289f4ae9206f914289f854498d644cc7dd109067e6" exitCode=7
Jul 15 16:45:03.251184 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:03.251024    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-439e5009-29735565-2kqsv" event={"ID":"a9d33864-ccf0-4aee-9ac8-50e292417eaa","Type":"ContainerDied","Data":"f4d043040b749d61ce81e9289f4ae9206f914289f854498d644cc7dd109067e6"}
Jul 15 16:45:03.251184 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:03.251077    2580 scope.go:117] "RemoveContainer" containerID="06e67e13cc15d46637a9d80ca46449e8988dc5f0a6bc9550db6b2b83336de343"
Jul 15 16:45:03.251570 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:03.251547    2580 scope.go:117] "RemoveContainer" containerID="f4d043040b749d61ce81e9289f4ae9206f914289f854498d644cc7dd109067e6"
Jul 15 16:45:03.251964 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:45:03.251936    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-migrate-439e5009-29735565-2kqsv_odh-ai-gateway-infra(a9d33864-ccf0-4aee-9ac8-50e292417eaa)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-439e5009-29735565-2kqsv" podUID="a9d33864-ccf0-4aee-9ac8-50e292417eaa"
Jul 15 16:45:03.398690 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:03.398655    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735565-4qmlx"
Jul 15 16:45:03.422971 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:03.422933    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:45:04.257830 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:04.257800    2580 scope.go:117] "RemoveContainer" containerID="f4d043040b749d61ce81e9289f4ae9206f914289f854498d644cc7dd109067e6"
Jul 15 16:45:04.258261 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:45:04.258041    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-migrate-439e5009-29735565-2kqsv_odh-ai-gateway-infra(a9d33864-ccf0-4aee-9ac8-50e292417eaa)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-439e5009-29735565-2kqsv" podUID="a9d33864-ccf0-4aee-9ac8-50e292417eaa"
Jul 15 16:45:04.258918 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:04.258892    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735565-4qmlx" event={"ID":"7eced033-1cb5-4264-9758-038529a62d22","Type":"ContainerDied","Data":"bb01199120943735dea0e495e6913ad11f6ad77eddfbce30d17a911bdf46add9"}
Jul 15 16:45:04.258965 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:04.258945    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb01199120943735dea0e495e6913ad11f6ad77eddfbce30d17a911bdf46add9"
Jul 15 16:45:04.258965 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:04.258904    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735565-4qmlx"
Jul 15 16:45:15.873254 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:15.873219    2580 scope.go:117] "RemoveContainer" containerID="fd9e4c5a58bb383d3476ae36d041a37d23561a532898802558b8cb337c60ed4e"
Jul 15 16:45:16.317968 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:16.317931    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr" event={"ID":"504b5c78-0f87-4a2b-bd61-2008d5650954","Type":"ContainerStarted","Data":"d20b098687e611674d50f55d8b1e0e55870d96673338bfc26770146263a7ffed"}
Jul 15 16:45:17.324093 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:17.324058    2580 generic.go:356] "Generic (PLEG): container finished" podID="504b5c78-0f87-4a2b-bd61-2008d5650954" containerID="d20b098687e611674d50f55d8b1e0e55870d96673338bfc26770146263a7ffed" exitCode=0
Jul 15 16:45:17.324532 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:17.324111    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr" event={"ID":"504b5c78-0f87-4a2b-bd61-2008d5650954","Type":"ContainerDied","Data":"d20b098687e611674d50f55d8b1e0e55870d96673338bfc26770146263a7ffed"}
Jul 15 16:45:17.324532 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:17.324164    2580 scope.go:117] "RemoveContainer" containerID="fd9e4c5a58bb383d3476ae36d041a37d23561a532898802558b8cb337c60ed4e"
Jul 15 16:45:17.353522 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:17.353491    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr"]
Jul 15 16:45:17.873396 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:17.873365    2580 scope.go:117] "RemoveContainer" containerID="f4d043040b749d61ce81e9289f4ae9206f914289f854498d644cc7dd109067e6"
Jul 15 16:45:18.331903 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:18.331874    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-439e5009-29735565-2kqsv" event={"ID":"a9d33864-ccf0-4aee-9ac8-50e292417eaa","Type":"ContainerStarted","Data":"f71997b55cc9e36c8b937131694da30034b261e2bcca7bd5036c3bec3a88840c"}
Jul 15 16:45:18.463218 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:18.463192    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr"
Jul 15 16:45:18.576057 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:18.576022    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/504b5c78-0f87-4a2b-bd61-2008d5650954-maas-api-service-ca\") pod \"504b5c78-0f87-4a2b-bd61-2008d5650954\" (UID: \"504b5c78-0f87-4a2b-bd61-2008d5650954\") "
Jul 15 16:45:18.576431 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:18.576406    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/504b5c78-0f87-4a2b-bd61-2008d5650954-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "504b5c78-0f87-4a2b-bd61-2008d5650954" (UID: "504b5c78-0f87-4a2b-bd61-2008d5650954"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:45:18.677525 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:18.677441    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/504b5c78-0f87-4a2b-bd61-2008d5650954-maas-api-service-ca\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:45:19.339064 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:19.339026    2580 generic.go:356] "Generic (PLEG): container finished" podID="a9d33864-ccf0-4aee-9ac8-50e292417eaa" containerID="f71997b55cc9e36c8b937131694da30034b261e2bcca7bd5036c3bec3a88840c" exitCode=0
Jul 15 16:45:19.339540 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:19.339096    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-439e5009-29735565-2kqsv" event={"ID":"a9d33864-ccf0-4aee-9ac8-50e292417eaa","Type":"ContainerDied","Data":"f71997b55cc9e36c8b937131694da30034b261e2bcca7bd5036c3bec3a88840c"}
Jul 15 16:45:19.339540 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:19.339160    2580 scope.go:117] "RemoveContainer" containerID="f4d043040b749d61ce81e9289f4ae9206f914289f854498d644cc7dd109067e6"
Jul 15 16:45:19.340651 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:19.340630    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr" event={"ID":"504b5c78-0f87-4a2b-bd61-2008d5650954","Type":"ContainerDied","Data":"9d875b86caf24e2102e2e3a84234f8479171850e8ba1d5f5410312e241e37c15"}
Jul 15 16:45:19.340726 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:19.340660    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr"
Jul 15 16:45:19.340726 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:19.340669    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d875b86caf24e2102e2e3a84234f8479171850e8ba1d5f5410312e241e37c15"
Jul 15 16:45:19.379372 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:19.379346    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-439e5009-29735565-2kqsv"]
Jul 15 16:45:19.386082 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:19.386056    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr"]
Jul 15 16:45:19.394184 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:19.394157    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-5s2gr"]
Jul 15 16:45:20.476484 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:20.476461    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-439e5009-29735565-2kqsv"
Jul 15 16:45:20.881351 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:20.881316    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="504b5c78-0f87-4a2b-bd61-2008d5650954" path="/var/lib/kubelet/pods/504b5c78-0f87-4a2b-bd61-2008d5650954/volumes"
Jul 15 16:45:21.353407 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:21.353376    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-439e5009-29735565-2kqsv"
Jul 15 16:45:21.353583 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:21.353376    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-439e5009-29735565-2kqsv" event={"ID":"a9d33864-ccf0-4aee-9ac8-50e292417eaa","Type":"ContainerDied","Data":"978f82739af7d2d80d75a951646f3d9c2ec09e11492c2a86ba58c19aeb5dd088"}
Jul 15 16:45:21.353583 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:21.353507    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="978f82739af7d2d80d75a951646f3d9c2ec09e11492c2a86ba58c19aeb5dd088"
Jul 15 16:45:21.383353 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:21.383317    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-439e5009-29735565-2kqsv"]
Jul 15 16:45:21.387842 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:21.387816    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-439e5009-29735565-2kqsv"]
Jul 15 16:45:22.878901 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:22.878864    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9d33864-ccf0-4aee-9ac8-50e292417eaa" path="/var/lib/kubelet/pods/a9d33864-ccf0-4aee-9ac8-50e292417eaa/volumes"
Jul 15 16:45:49.922305 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:49.922267    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-69jj9"]
Jul 15 16:45:49.922852 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:49.922741    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9d33864-ccf0-4aee-9ac8-50e292417eaa" containerName="cleanup"
Jul 15 16:45:49.922852 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:49.922779    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9d33864-ccf0-4aee-9ac8-50e292417eaa" containerName="cleanup"
Jul 15 16:45:49.922852 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:49.922818    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9d33864-ccf0-4aee-9ac8-50e292417eaa" containerName="cleanup"
Jul 15 16:45:49.922852 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:49.922827    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9d33864-ccf0-4aee-9ac8-50e292417eaa" containerName="cleanup"
Jul 15 16:45:49.922852 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:49.922837    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="504b5c78-0f87-4a2b-bd61-2008d5650954" containerName="revoke-keys"
Jul 15 16:45:49.922852 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:49.922845    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="504b5c78-0f87-4a2b-bd61-2008d5650954" containerName="revoke-keys"
Jul 15 16:45:49.922852 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:49.922854    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9d33864-ccf0-4aee-9ac8-50e292417eaa" containerName="cleanup"
Jul 15 16:45:49.923204 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:49.922862    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9d33864-ccf0-4aee-9ac8-50e292417eaa" containerName="cleanup"
Jul 15 16:45:49.923204 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:49.922874    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="504b5c78-0f87-4a2b-bd61-2008d5650954" containerName="revoke-keys"
Jul 15 16:45:49.923204 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:49.922882    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="504b5c78-0f87-4a2b-bd61-2008d5650954" containerName="revoke-keys"
Jul 15 16:45:49.923204 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:49.922895    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eced033-1cb5-4264-9758-038529a62d22" containerName="cleanup"
Jul 15 16:45:49.923204 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:49.922903    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eced033-1cb5-4264-9758-038529a62d22" containerName="cleanup"
Jul 15 16:45:49.923204 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:49.922990    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="504b5c78-0f87-4a2b-bd61-2008d5650954" containerName="revoke-keys"
Jul 15 16:45:49.923204 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:49.923002    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9d33864-ccf0-4aee-9ac8-50e292417eaa" containerName="cleanup"
Jul 15 16:45:49.923204 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:49.923010    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="504b5c78-0f87-4a2b-bd61-2008d5650954" containerName="revoke-keys"
Jul 15 16:45:49.923204 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:49.923020    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="504b5c78-0f87-4a2b-bd61-2008d5650954" containerName="revoke-keys"
Jul 15 16:45:49.923204 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:49.923028    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7eced033-1cb5-4264-9758-038529a62d22" containerName="cleanup"
Jul 15 16:45:49.923204 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:49.923037    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9d33864-ccf0-4aee-9ac8-50e292417eaa" containerName="cleanup"
Jul 15 16:45:49.923204 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:49.923045    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9d33864-ccf0-4aee-9ac8-50e292417eaa" containerName="cleanup"
Jul 15 16:45:49.926222 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:49.926202    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-69jj9"
Jul 15 16:45:49.929090 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:49.929068    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lsp65\""
Jul 15 16:45:49.929191 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:49.929105    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 16:45:49.934170 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:49.934146    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-69jj9"]
Jul 15 16:45:50.057001 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:50.056965    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/deccb51a-4f6a-4404-a271-6a5456d7fda2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-439e5009-69jj9\" (UID: \"deccb51a-4f6a-4404-a271-6a5456d7fda2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-69jj9"
Jul 15 16:45:50.157463 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:50.157431    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/deccb51a-4f6a-4404-a271-6a5456d7fda2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-439e5009-69jj9\" (UID: \"deccb51a-4f6a-4404-a271-6a5456d7fda2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-69jj9"
Jul 15 16:45:50.158051 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:50.158034    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/deccb51a-4f6a-4404-a271-6a5456d7fda2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-439e5009-69jj9\" (UID: \"deccb51a-4f6a-4404-a271-6a5456d7fda2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-69jj9"
Jul 15 16:45:50.237604 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:50.237533    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-69jj9"
Jul 15 16:45:50.372825 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:50.372798    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-69jj9"]
Jul 15 16:45:50.374630 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:45:50.374599    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeccb51a_4f6a_4404_a271_6a5456d7fda2.slice/crio-4b63105db4bfb4255c870dcbba71223f8c397ee927c3e95d6e965d3b284a895b WatchSource:0}: Error finding container 4b63105db4bfb4255c870dcbba71223f8c397ee927c3e95d6e965d3b284a895b: Status 404 returned error can't find the container with id 4b63105db4bfb4255c870dcbba71223f8c397ee927c3e95d6e965d3b284a895b
Jul 15 16:45:50.484999 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:50.484969    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-69jj9" event={"ID":"deccb51a-4f6a-4404-a271-6a5456d7fda2","Type":"ContainerStarted","Data":"4b63105db4bfb4255c870dcbba71223f8c397ee927c3e95d6e965d3b284a895b"}
Jul 15 16:45:51.491569 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:51.491532    2580 generic.go:356] "Generic (PLEG): container finished" podID="deccb51a-4f6a-4404-a271-6a5456d7fda2" containerID="55d5e347e8aef4402b609f8776183134d4a0b55445040560b0a09a2627ab3598" exitCode=0
Jul 15 16:45:51.492024 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:51.491610    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-69jj9" event={"ID":"deccb51a-4f6a-4404-a271-6a5456d7fda2","Type":"ContainerDied","Data":"55d5e347e8aef4402b609f8776183134d4a0b55445040560b0a09a2627ab3598"}
Jul 15 16:45:52.633543 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:52.633514    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-69jj9"
Jul 15 16:45:52.678373 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:52.678339    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/deccb51a-4f6a-4404-a271-6a5456d7fda2-maas-api-service-ca\") pod \"deccb51a-4f6a-4404-a271-6a5456d7fda2\" (UID: \"deccb51a-4f6a-4404-a271-6a5456d7fda2\") "
Jul 15 16:45:52.678717 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:52.678693    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/deccb51a-4f6a-4404-a271-6a5456d7fda2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "deccb51a-4f6a-4404-a271-6a5456d7fda2" (UID: "deccb51a-4f6a-4404-a271-6a5456d7fda2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:45:52.779083 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:52.778992    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/deccb51a-4f6a-4404-a271-6a5456d7fda2-maas-api-service-ca\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:45:53.504095 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:53.504065    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-69jj9"
Jul 15 16:45:53.504095 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:53.504095    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-439e5009-69jj9" event={"ID":"deccb51a-4f6a-4404-a271-6a5456d7fda2","Type":"ContainerDied","Data":"4b63105db4bfb4255c870dcbba71223f8c397ee927c3e95d6e965d3b284a895b"}
Jul 15 16:45:53.504357 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:45:53.504137    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4b63105db4bfb4255c870dcbba71223f8c397ee927c3e95d6e965d3b284a895b"
Jul 15 16:46:00.022357 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:00.022304    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:46:00.139047 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:00.139009    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:46:00.631752 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:00.631712    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:46:03.083963 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:03.083923    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:46:06.586033 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:06.585993    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:46:07.807635 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:07.807593    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:46:07.908001 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:07.907961    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:46:08.235129 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:08.235033    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:46:08.331381 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:08.331339    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:46:08.432246 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:08.432200    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:46:11.365151 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:11.365110    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:46:12.782843 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:12.782808    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg"]
Jul 15 16:46:12.783311 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:12.783133    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="deccb51a-4f6a-4404-a271-6a5456d7fda2" containerName="revoke-keys"
Jul 15 16:46:12.783311 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:12.783144    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="deccb51a-4f6a-4404-a271-6a5456d7fda2" containerName="revoke-keys"
Jul 15 16:46:12.783311 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:12.783156    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="504b5c78-0f87-4a2b-bd61-2008d5650954" containerName="revoke-keys"
Jul 15 16:46:12.783311 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:12.783161    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="504b5c78-0f87-4a2b-bd61-2008d5650954" containerName="revoke-keys"
Jul 15 16:46:12.783311 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:12.783226    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="deccb51a-4f6a-4404-a271-6a5456d7fda2" containerName="revoke-keys"
Jul 15 16:46:12.786019 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:12.786002    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg"
Jul 15 16:46:12.789628 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:12.789606    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lsp65\""
Jul 15 16:46:12.790004 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:12.789986    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 16:46:12.800016 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:12.799993    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg"]
Jul 15 16:46:12.952317 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:12.952276    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a7c82eb-4786-4265-aa40-d4c236990f58-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg\" (UID: \"3a7c82eb-4786-4265-aa40-d4c236990f58\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg"
Jul 15 16:46:13.052795 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:13.052734    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a7c82eb-4786-4265-aa40-d4c236990f58-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg\" (UID: \"3a7c82eb-4786-4265-aa40-d4c236990f58\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg"
Jul 15 16:46:13.053427 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:13.053401    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a7c82eb-4786-4265-aa40-d4c236990f58-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg\" (UID: \"3a7c82eb-4786-4265-aa40-d4c236990f58\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg"
Jul 15 16:46:13.096478 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:13.096442    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg"
Jul 15 16:46:13.242367 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:13.242341    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg"]
Jul 15 16:46:13.244470 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:46:13.244440    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a7c82eb_4786_4265_aa40_d4c236990f58.slice/crio-9d4e0a9a46780253b972a2fb0b5c7c14cacb6dd0bd1eb849edb86de37e18ab2f WatchSource:0}: Error finding container 9d4e0a9a46780253b972a2fb0b5c7c14cacb6dd0bd1eb849edb86de37e18ab2f: Status 404 returned error can't find the container with id 9d4e0a9a46780253b972a2fb0b5c7c14cacb6dd0bd1eb849edb86de37e18ab2f
Jul 15 16:46:13.597336 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:13.597301    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg" event={"ID":"3a7c82eb-4786-4265-aa40-d4c236990f58","Type":"ContainerStarted","Data":"c3748f634edf79885456bccf81cd31c816e4732b4e937d31e464623f660ae4c9"}
Jul 15 16:46:13.597523 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:13.597352    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg" event={"ID":"3a7c82eb-4786-4265-aa40-d4c236990f58","Type":"ContainerStarted","Data":"9d4e0a9a46780253b972a2fb0b5c7c14cacb6dd0bd1eb849edb86de37e18ab2f"}
Jul 15 16:46:13.612970 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:13.612916    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg" podStartSLOduration=1.612897523 podStartE2EDuration="1.612897523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:46:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:46:13.61243342 +0000 UTC m=+1775.259473957" watchObservedRunningTime="2026-07-15 16:46:13.612897523 +0000 UTC m=+1775.259938083"
Jul 15 16:46:14.605601 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:14.605565    2580 generic.go:356] "Generic (PLEG): container finished" podID="3a7c82eb-4786-4265-aa40-d4c236990f58" containerID="c3748f634edf79885456bccf81cd31c816e4732b4e937d31e464623f660ae4c9" exitCode=7
Jul 15 16:46:14.606108 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:14.605652    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg" event={"ID":"3a7c82eb-4786-4265-aa40-d4c236990f58","Type":"ContainerDied","Data":"c3748f634edf79885456bccf81cd31c816e4732b4e937d31e464623f660ae4c9"}
Jul 15 16:46:14.606108 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:14.605885    2580 scope.go:117] "RemoveContainer" containerID="c3748f634edf79885456bccf81cd31c816e4732b4e937d31e464623f660ae4c9"
Jul 15 16:46:15.613031 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:15.612941    2580 generic.go:356] "Generic (PLEG): container finished" podID="3a7c82eb-4786-4265-aa40-d4c236990f58" containerID="5688ce733d0b90bf305075c9041a10de37efc1cc37b4028aaba81aaaab729b2b" exitCode=7
Jul 15 16:46:15.613031 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:15.613004    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg" event={"ID":"3a7c82eb-4786-4265-aa40-d4c236990f58","Type":"ContainerDied","Data":"5688ce733d0b90bf305075c9041a10de37efc1cc37b4028aaba81aaaab729b2b"}
Jul 15 16:46:15.613530 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:15.613047    2580 scope.go:117] "RemoveContainer" containerID="c3748f634edf79885456bccf81cd31c816e4732b4e937d31e464623f660ae4c9"
Jul 15 16:46:15.613530 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:15.613238    2580 scope.go:117] "RemoveContainer" containerID="5688ce733d0b90bf305075c9041a10de37efc1cc37b4028aaba81aaaab729b2b"
Jul 15 16:46:15.613530 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:46:15.613463    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg_odh-ai-gateway-infra(3a7c82eb-4786-4265-aa40-d4c236990f58)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg" podUID="3a7c82eb-4786-4265-aa40-d4c236990f58"
Jul 15 16:46:16.291536 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:16.291499    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:46:16.619165 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:16.619134    2580 scope.go:117] "RemoveContainer" containerID="5688ce733d0b90bf305075c9041a10de37efc1cc37b4028aaba81aaaab729b2b"
Jul 15 16:46:16.619632 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:46:16.619371    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg_odh-ai-gateway-infra(3a7c82eb-4786-4265-aa40-d4c236990f58)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg" podUID="3a7c82eb-4786-4265-aa40-d4c236990f58"
Jul 15 16:46:29.873912 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:29.873879    2580 scope.go:117] "RemoveContainer" containerID="5688ce733d0b90bf305075c9041a10de37efc1cc37b4028aaba81aaaab729b2b"
Jul 15 16:46:30.687193 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:30.687157    2580 generic.go:356] "Generic (PLEG): container finished" podID="3a7c82eb-4786-4265-aa40-d4c236990f58" containerID="b3142d4aa5cdbefdd921e0cb7d96c263914f79271879f396025b54e04d85086d" exitCode=0
Jul 15 16:46:30.687380 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:30.687220    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg" event={"ID":"3a7c82eb-4786-4265-aa40-d4c236990f58","Type":"ContainerDied","Data":"b3142d4aa5cdbefdd921e0cb7d96c263914f79271879f396025b54e04d85086d"}
Jul 15 16:46:30.687380 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:30.687270    2580 scope.go:117] "RemoveContainer" containerID="5688ce733d0b90bf305075c9041a10de37efc1cc37b4028aaba81aaaab729b2b"
Jul 15 16:46:31.718808 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:31.718751    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg"]
Jul 15 16:46:31.845923 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:31.845893    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg"
Jul 15 16:46:31.924405 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:31.924365    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a7c82eb-4786-4265-aa40-d4c236990f58-maas-api-service-ca\") pod \"3a7c82eb-4786-4265-aa40-d4c236990f58\" (UID: \"3a7c82eb-4786-4265-aa40-d4c236990f58\") "
Jul 15 16:46:31.924740 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:31.924721    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a7c82eb-4786-4265-aa40-d4c236990f58-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3a7c82eb-4786-4265-aa40-d4c236990f58" (UID: "3a7c82eb-4786-4265-aa40-d4c236990f58"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:46:32.025606 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:32.025515    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a7c82eb-4786-4265-aa40-d4c236990f58-maas-api-service-ca\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:46:32.699207 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:32.699176    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg"
Jul 15 16:46:32.699420 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:32.699219    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg" event={"ID":"3a7c82eb-4786-4265-aa40-d4c236990f58","Type":"ContainerDied","Data":"9d4e0a9a46780253b972a2fb0b5c7c14cacb6dd0bd1eb849edb86de37e18ab2f"}
Jul 15 16:46:32.699420 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:32.699280    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d4e0a9a46780253b972a2fb0b5c7c14cacb6dd0bd1eb849edb86de37e18ab2f"
Jul 15 16:46:32.731942 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:32.731909    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg"]
Jul 15 16:46:32.736019 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:32.735996    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-d54gg"]
Jul 15 16:46:32.878905 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:32.878866    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a7c82eb-4786-4265-aa40-d4c236990f58" path="/var/lib/kubelet/pods/3a7c82eb-4786-4265-aa40-d4c236990f58/volumes"
Jul 15 16:46:37.187697 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:37.187664    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:46:37.284557 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:37.284510    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:46:38.845549 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:38.845516    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-295d8_46356463-bfb1-42b8-8871-71357489a95b/ovn-acl-logging/0.log"
Jul 15 16:46:38.848165 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:46:38.848144    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-295d8_46356463-bfb1-42b8-8871-71357489a95b/ovn-acl-logging/0.log"
Jul 15 16:47:02.823621 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:02.823586    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-b99cz"]
Jul 15 16:47:02.824101 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:02.823988    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a7c82eb-4786-4265-aa40-d4c236990f58" containerName="revoke-keys"
Jul 15 16:47:02.824101 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:02.824002    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a7c82eb-4786-4265-aa40-d4c236990f58" containerName="revoke-keys"
Jul 15 16:47:02.824101 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:02.824019    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a7c82eb-4786-4265-aa40-d4c236990f58" containerName="revoke-keys"
Jul 15 16:47:02.824101 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:02.824024    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a7c82eb-4786-4265-aa40-d4c236990f58" containerName="revoke-keys"
Jul 15 16:47:02.824101 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:02.824086    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a7c82eb-4786-4265-aa40-d4c236990f58" containerName="revoke-keys"
Jul 15 16:47:02.824101 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:02.824099    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a7c82eb-4786-4265-aa40-d4c236990f58" containerName="revoke-keys"
Jul 15 16:47:02.827341 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:02.827325    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-b99cz"
Jul 15 16:47:02.830157 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:02.830132    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lsp65\""
Jul 15 16:47:02.830302 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:02.830134    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 16:47:02.837068 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:02.837042    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-b99cz"]
Jul 15 16:47:02.990209 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:02.990172    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ec7c91b-db8b-48ba-acd0-c6feb8ba7ef6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3ac1ded3-b99cz\" (UID: \"9ec7c91b-db8b-48ba-acd0-c6feb8ba7ef6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-b99cz"
Jul 15 16:47:03.091567 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:03.091478    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ec7c91b-db8b-48ba-acd0-c6feb8ba7ef6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3ac1ded3-b99cz\" (UID: \"9ec7c91b-db8b-48ba-acd0-c6feb8ba7ef6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-b99cz"
Jul 15 16:47:03.092162 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:03.092142    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ec7c91b-db8b-48ba-acd0-c6feb8ba7ef6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3ac1ded3-b99cz\" (UID: \"9ec7c91b-db8b-48ba-acd0-c6feb8ba7ef6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-b99cz"
Jul 15 16:47:03.139297 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:03.139269    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-b99cz"
Jul 15 16:47:03.269336 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:03.269310    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-b99cz"]
Jul 15 16:47:03.271458 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:47:03.271414    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ec7c91b_db8b_48ba_acd0_c6feb8ba7ef6.slice/crio-4ca91f2b6a3c07c443e76e1dc548885f672b7f473cb1e362375cb01dfbbaf711 WatchSource:0}: Error finding container 4ca91f2b6a3c07c443e76e1dc548885f672b7f473cb1e362375cb01dfbbaf711: Status 404 returned error can't find the container with id 4ca91f2b6a3c07c443e76e1dc548885f672b7f473cb1e362375cb01dfbbaf711
Jul 15 16:47:03.412374 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:03.412338    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:47:03.510518 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:03.510483    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:47:03.845403 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:03.845376    2580 generic.go:356] "Generic (PLEG): container finished" podID="9ec7c91b-db8b-48ba-acd0-c6feb8ba7ef6" containerID="ca1f1df44ced033a5db8d3a6908dd7def052b5425d9efa23b99507cb1435c33c" exitCode=0
Jul 15 16:47:03.845740 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:03.845468    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-b99cz" event={"ID":"9ec7c91b-db8b-48ba-acd0-c6feb8ba7ef6","Type":"ContainerDied","Data":"ca1f1df44ced033a5db8d3a6908dd7def052b5425d9efa23b99507cb1435c33c"}
Jul 15 16:47:03.845740 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:03.845542    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-b99cz" event={"ID":"9ec7c91b-db8b-48ba-acd0-c6feb8ba7ef6","Type":"ContainerStarted","Data":"4ca91f2b6a3c07c443e76e1dc548885f672b7f473cb1e362375cb01dfbbaf711"}
Jul 15 16:47:04.978277 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:04.978253    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-b99cz"
Jul 15 16:47:05.109391 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:05.109291    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ec7c91b-db8b-48ba-acd0-c6feb8ba7ef6-maas-api-service-ca\") pod \"9ec7c91b-db8b-48ba-acd0-c6feb8ba7ef6\" (UID: \"9ec7c91b-db8b-48ba-acd0-c6feb8ba7ef6\") "
Jul 15 16:47:05.109662 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:05.109638    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ec7c91b-db8b-48ba-acd0-c6feb8ba7ef6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9ec7c91b-db8b-48ba-acd0-c6feb8ba7ef6" (UID: "9ec7c91b-db8b-48ba-acd0-c6feb8ba7ef6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:47:05.210017 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:05.209982    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ec7c91b-db8b-48ba-acd0-c6feb8ba7ef6-maas-api-service-ca\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:47:05.855403 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:05.855371    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-b99cz"
Jul 15 16:47:05.855403 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:05.855396    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ac1ded3-b99cz" event={"ID":"9ec7c91b-db8b-48ba-acd0-c6feb8ba7ef6","Type":"ContainerDied","Data":"4ca91f2b6a3c07c443e76e1dc548885f672b7f473cb1e362375cb01dfbbaf711"}
Jul 15 16:47:05.855672 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:05.855463    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ca91f2b6a3c07c443e76e1dc548885f672b7f473cb1e362375cb01dfbbaf711"
Jul 15 16:47:13.506894 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:13.506851    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:47:16.783048 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:16.783008    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:47:17.917032 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:17.916988    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:47:23.772136 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:23.772098    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:47:23.836524 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:23.836489    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:47:29.049242 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:29.049203    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:47:29.367590 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:29.367552    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:47:30.504647 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:30.504611    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:47:30.604957 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:30.604921    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:47:30.682154 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:30.682122    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:47:30.967422 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:30.967383    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:47:31.188088 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:31.188051    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:47:31.288579 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:31.288484    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:47:34.885432 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:34.885399    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-mcc59"]
Jul 15 16:47:34.885917 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:34.885717    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ec7c91b-db8b-48ba-acd0-c6feb8ba7ef6" containerName="revoke-keys"
Jul 15 16:47:34.885917 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:34.885727    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ec7c91b-db8b-48ba-acd0-c6feb8ba7ef6" containerName="revoke-keys"
Jul 15 16:47:34.885917 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:34.885740    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a7c82eb-4786-4265-aa40-d4c236990f58" containerName="revoke-keys"
Jul 15 16:47:34.885917 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:34.885746    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a7c82eb-4786-4265-aa40-d4c236990f58" containerName="revoke-keys"
Jul 15 16:47:34.885917 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:34.885811    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ec7c91b-db8b-48ba-acd0-c6feb8ba7ef6" containerName="revoke-keys"
Jul 15 16:47:34.885917 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:34.885824    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a7c82eb-4786-4265-aa40-d4c236990f58" containerName="revoke-keys"
Jul 15 16:47:34.889160 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:34.889140    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-mcc59"
Jul 15 16:47:34.891742 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:34.891722    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 16:47:34.891742 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:34.891734    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lsp65\""
Jul 15 16:47:34.897179 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:34.897146    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-mcc59"]
Jul 15 16:47:34.954310 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:34.954263    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0de72e8e-c806-4c04-8dc3-91a2e6b2871b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-77b034d1-mcc59\" (UID: \"0de72e8e-c806-4c04-8dc3-91a2e6b2871b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-mcc59"
Jul 15 16:47:35.055511 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:35.055473    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0de72e8e-c806-4c04-8dc3-91a2e6b2871b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-77b034d1-mcc59\" (UID: \"0de72e8e-c806-4c04-8dc3-91a2e6b2871b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-mcc59"
Jul 15 16:47:35.056145 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:35.056124    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0de72e8e-c806-4c04-8dc3-91a2e6b2871b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-77b034d1-mcc59\" (UID: \"0de72e8e-c806-4c04-8dc3-91a2e6b2871b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-mcc59"
Jul 15 16:47:35.200301 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:35.200222    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-mcc59"
Jul 15 16:47:35.330113 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:35.330083    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-mcc59"]
Jul 15 16:47:35.333215 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:47:35.333182    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0de72e8e_c806_4c04_8dc3_91a2e6b2871b.slice/crio-14085aa4c4506983e07fcc90c288c37e70f6321b18ef115599dfdf33ce101782 WatchSource:0}: Error finding container 14085aa4c4506983e07fcc90c288c37e70f6321b18ef115599dfdf33ce101782: Status 404 returned error can't find the container with id 14085aa4c4506983e07fcc90c288c37e70f6321b18ef115599dfdf33ce101782
Jul 15 16:47:35.476498 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:35.476413    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:47:35.574911 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:35.574881    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:47:36.003697 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:36.003662    2580 generic.go:356] "Generic (PLEG): container finished" podID="0de72e8e-c806-4c04-8dc3-91a2e6b2871b" containerID="4c44aca86ac09fb506cc83777714375d3cd7372fef6babaf91d9651375fe616b" exitCode=7
Jul 15 16:47:36.004105 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:36.003741    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-mcc59" event={"ID":"0de72e8e-c806-4c04-8dc3-91a2e6b2871b","Type":"ContainerDied","Data":"4c44aca86ac09fb506cc83777714375d3cd7372fef6babaf91d9651375fe616b"}
Jul 15 16:47:36.004105 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:36.003809    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-mcc59" event={"ID":"0de72e8e-c806-4c04-8dc3-91a2e6b2871b","Type":"ContainerStarted","Data":"14085aa4c4506983e07fcc90c288c37e70f6321b18ef115599dfdf33ce101782"}
Jul 15 16:47:36.004105 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:36.003988    2580 scope.go:117] "RemoveContainer" containerID="4c44aca86ac09fb506cc83777714375d3cd7372fef6babaf91d9651375fe616b"
Jul 15 16:47:37.010512 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:37.010425    2580 generic.go:356] "Generic (PLEG): container finished" podID="0de72e8e-c806-4c04-8dc3-91a2e6b2871b" containerID="21264c0d257909166c3ff1c0e271b088fbc551728a190692a8cf59d99bcf0ac4" exitCode=7
Jul 15 16:47:37.010936 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:37.010507    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-mcc59" event={"ID":"0de72e8e-c806-4c04-8dc3-91a2e6b2871b","Type":"ContainerDied","Data":"21264c0d257909166c3ff1c0e271b088fbc551728a190692a8cf59d99bcf0ac4"}
Jul 15 16:47:37.010936 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:37.010567    2580 scope.go:117] "RemoveContainer" containerID="4c44aca86ac09fb506cc83777714375d3cd7372fef6babaf91d9651375fe616b"
Jul 15 16:47:37.010936 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:37.010818    2580 scope.go:117] "RemoveContainer" containerID="21264c0d257909166c3ff1c0e271b088fbc551728a190692a8cf59d99bcf0ac4"
Jul 15 16:47:37.011086 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:47:37.011026    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-77b034d1-mcc59_odh-ai-gateway-infra(0de72e8e-c806-4c04-8dc3-91a2e6b2871b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-mcc59" podUID="0de72e8e-c806-4c04-8dc3-91a2e6b2871b"
Jul 15 16:47:38.017084 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:38.017048    2580 scope.go:117] "RemoveContainer" containerID="21264c0d257909166c3ff1c0e271b088fbc551728a190692a8cf59d99bcf0ac4"
Jul 15 16:47:38.017486 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:47:38.017230    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-77b034d1-mcc59_odh-ai-gateway-infra(0de72e8e-c806-4c04-8dc3-91a2e6b2871b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-mcc59" podUID="0de72e8e-c806-4c04-8dc3-91a2e6b2871b"
Jul 15 16:47:38.750622 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:38.750589    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:47:39.114925 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:39.114886    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:47:52.873901 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:52.873862    2580 scope.go:117] "RemoveContainer" containerID="21264c0d257909166c3ff1c0e271b088fbc551728a190692a8cf59d99bcf0ac4"
Jul 15 16:47:54.098621 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:54.098587    2580 generic.go:356] "Generic (PLEG): container finished" podID="0de72e8e-c806-4c04-8dc3-91a2e6b2871b" containerID="18e295ffbbc9cdcea5097b484b1cb573b145b131fbe4bb6d80f43bd85f52f985" exitCode=0
Jul 15 16:47:54.099115 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:54.098667    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-mcc59" event={"ID":"0de72e8e-c806-4c04-8dc3-91a2e6b2871b","Type":"ContainerDied","Data":"18e295ffbbc9cdcea5097b484b1cb573b145b131fbe4bb6d80f43bd85f52f985"}
Jul 15 16:47:54.099115 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:54.098740    2580 scope.go:117] "RemoveContainer" containerID="21264c0d257909166c3ff1c0e271b088fbc551728a190692a8cf59d99bcf0ac4"
Jul 15 16:47:55.127709 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:55.127676    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-mcc59"]
Jul 15 16:47:55.252530 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:55.252504    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-mcc59"
Jul 15 16:47:55.335635 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:55.335593    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0de72e8e-c806-4c04-8dc3-91a2e6b2871b-maas-api-service-ca\") pod \"0de72e8e-c806-4c04-8dc3-91a2e6b2871b\" (UID: \"0de72e8e-c806-4c04-8dc3-91a2e6b2871b\") "
Jul 15 16:47:55.336046 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:55.336021    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0de72e8e-c806-4c04-8dc3-91a2e6b2871b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0de72e8e-c806-4c04-8dc3-91a2e6b2871b" (UID: "0de72e8e-c806-4c04-8dc3-91a2e6b2871b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:47:55.436207 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:55.436113    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0de72e8e-c806-4c04-8dc3-91a2e6b2871b-maas-api-service-ca\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:47:56.111157 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:56.111116    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-mcc59"
Jul 15 16:47:56.111337 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:56.111159    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-mcc59" event={"ID":"0de72e8e-c806-4c04-8dc3-91a2e6b2871b","Type":"ContainerDied","Data":"14085aa4c4506983e07fcc90c288c37e70f6321b18ef115599dfdf33ce101782"}
Jul 15 16:47:56.111337 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:56.111210    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="14085aa4c4506983e07fcc90c288c37e70f6321b18ef115599dfdf33ce101782"
Jul 15 16:47:56.145110 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:56.145079    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-mcc59"]
Jul 15 16:47:56.146659 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:56.146639    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-mcc59"]
Jul 15 16:47:56.884662 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:47:56.884625    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0de72e8e-c806-4c04-8dc3-91a2e6b2871b" path="/var/lib/kubelet/pods/0de72e8e-c806-4c04-8dc3-91a2e6b2871b/volumes"
Jul 15 16:48:34.941178 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:34.941098    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-scdnv"]
Jul 15 16:48:34.941618 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:34.941415    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0de72e8e-c806-4c04-8dc3-91a2e6b2871b" containerName="revoke-keys"
Jul 15 16:48:34.941618 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:34.941426    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0de72e8e-c806-4c04-8dc3-91a2e6b2871b" containerName="revoke-keys"
Jul 15 16:48:34.941618 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:34.941444    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0de72e8e-c806-4c04-8dc3-91a2e6b2871b" containerName="revoke-keys"
Jul 15 16:48:34.941618 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:34.941449    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0de72e8e-c806-4c04-8dc3-91a2e6b2871b" containerName="revoke-keys"
Jul 15 16:48:34.941618 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:34.941458    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0de72e8e-c806-4c04-8dc3-91a2e6b2871b" containerName="revoke-keys"
Jul 15 16:48:34.941618 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:34.941463    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0de72e8e-c806-4c04-8dc3-91a2e6b2871b" containerName="revoke-keys"
Jul 15 16:48:34.941618 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:34.941524    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0de72e8e-c806-4c04-8dc3-91a2e6b2871b" containerName="revoke-keys"
Jul 15 16:48:34.941618 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:34.941534    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0de72e8e-c806-4c04-8dc3-91a2e6b2871b" containerName="revoke-keys"
Jul 15 16:48:34.941618 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:34.941541    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0de72e8e-c806-4c04-8dc3-91a2e6b2871b" containerName="revoke-keys"
Jul 15 16:48:34.944644 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:34.944625    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-scdnv"
Jul 15 16:48:34.947128 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:34.947107    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lsp65\""
Jul 15 16:48:34.947238 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:34.947108    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 16:48:34.954141 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:34.954113    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-scdnv"]
Jul 15 16:48:35.076880 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:35.076838    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02ceffc6-50fd-4ef8-96db-45ff4e0cd4e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-77b034d1-scdnv\" (UID: \"02ceffc6-50fd-4ef8-96db-45ff4e0cd4e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-scdnv"
Jul 15 16:48:35.177466 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:35.177425    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02ceffc6-50fd-4ef8-96db-45ff4e0cd4e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-77b034d1-scdnv\" (UID: \"02ceffc6-50fd-4ef8-96db-45ff4e0cd4e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-scdnv"
Jul 15 16:48:35.178076 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:35.178057    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02ceffc6-50fd-4ef8-96db-45ff4e0cd4e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-77b034d1-scdnv\" (UID: \"02ceffc6-50fd-4ef8-96db-45ff4e0cd4e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-scdnv"
Jul 15 16:48:35.256592 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:35.256500    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-scdnv"
Jul 15 16:48:35.389938 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:35.389914    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-scdnv"]
Jul 15 16:48:35.392390 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:48:35.392355    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02ceffc6_50fd_4ef8_96db_45ff4e0cd4e2.slice/crio-14ea43a83310fc385bb0308f70d5e16ffc9889bc6741a3854958faef1c1713ac WatchSource:0}: Error finding container 14ea43a83310fc385bb0308f70d5e16ffc9889bc6741a3854958faef1c1713ac: Status 404 returned error can't find the container with id 14ea43a83310fc385bb0308f70d5e16ffc9889bc6741a3854958faef1c1713ac
Jul 15 16:48:36.313998 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:36.313958    2580 generic.go:356] "Generic (PLEG): container finished" podID="02ceffc6-50fd-4ef8-96db-45ff4e0cd4e2" containerID="7430ac80ae26435c1602e28bac9cf7e2c7314c1560f3a2c5a30e516f9638b267" exitCode=0
Jul 15 16:48:36.314389 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:36.314013    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-scdnv" event={"ID":"02ceffc6-50fd-4ef8-96db-45ff4e0cd4e2","Type":"ContainerDied","Data":"7430ac80ae26435c1602e28bac9cf7e2c7314c1560f3a2c5a30e516f9638b267"}
Jul 15 16:48:36.314389 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:36.314095    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-scdnv" event={"ID":"02ceffc6-50fd-4ef8-96db-45ff4e0cd4e2","Type":"ContainerStarted","Data":"14ea43a83310fc385bb0308f70d5e16ffc9889bc6741a3854958faef1c1713ac"}
Jul 15 16:48:37.457667 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:37.457637    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-scdnv"
Jul 15 16:48:37.599001 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:37.598909    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02ceffc6-50fd-4ef8-96db-45ff4e0cd4e2-maas-api-service-ca\") pod \"02ceffc6-50fd-4ef8-96db-45ff4e0cd4e2\" (UID: \"02ceffc6-50fd-4ef8-96db-45ff4e0cd4e2\") "
Jul 15 16:48:37.599309 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:37.599286    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02ceffc6-50fd-4ef8-96db-45ff4e0cd4e2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "02ceffc6-50fd-4ef8-96db-45ff4e0cd4e2" (UID: "02ceffc6-50fd-4ef8-96db-45ff4e0cd4e2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:48:37.700140 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:37.700105    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02ceffc6-50fd-4ef8-96db-45ff4e0cd4e2-maas-api-service-ca\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:48:38.323885 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:38.323847    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-scdnv"
Jul 15 16:48:38.324059 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:38.323894    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-77b034d1-scdnv" event={"ID":"02ceffc6-50fd-4ef8-96db-45ff4e0cd4e2","Type":"ContainerDied","Data":"14ea43a83310fc385bb0308f70d5e16ffc9889bc6741a3854958faef1c1713ac"}
Jul 15 16:48:38.324059 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:38.323942    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="14ea43a83310fc385bb0308f70d5e16ffc9889bc6741a3854958faef1c1713ac"
Jul 15 16:48:45.485494 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:45.485450    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:48:48.688213 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:48.688179    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:48:50.032224 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:50.032184    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:48:55.767236 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:55.767198    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:48:55.831943 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:55.831911    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:48:58.664904 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:48:58.664859    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:49:00.244905 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:49:00.244869    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:49:00.890818 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:49:00.890784    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:49:00.989058 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:49:00.989019    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:49:01.083210 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:49:01.083168    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:49:02.647989 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:49:02.647949    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:49:06.769702 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:49:06.769667    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:49:08.527372 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:49:08.527332    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:49:08.635110 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:49:08.635069    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:49:08.774681 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:49:08.774640    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:49:08.940465 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:49:08.940424    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:49:09.358295 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:49:09.358254    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:49:35.360970 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:49:35.360882    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:49:41.578477 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:49:41.578438    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:49:43.307422 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:49:43.307368    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:49:57.500918 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:49:57.500875    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:03.972876 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:03.972830    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:04.282096 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:04.282000    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:04.382001 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:04.381949    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:14.770206 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:14.770169    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:15.662935 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:15.662894    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:21.676441 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:21.676400    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:21.781790 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:21.781740    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:22.004037 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:22.003938    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:22.138993 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:22.138957    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:23.320840 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:23.320783    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:23.422654 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:23.422618    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:24.194407 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:24.194369    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:50:25.011593 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:25.011554    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:30.957945 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:30.957911    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:31.056445 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:31.056411    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:31.648153 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:31.648119    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:32.083339 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:32.083298    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:50:37.666420 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:37.666380    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:37.768101 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:37.768053    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:37.932196 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:37.932094    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:38.862681 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:38.862641    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:38.926777 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:38.926721    2580 scope.go:117] "RemoveContainer" containerID="902b6aaa1a549196b9e1a43351c26919fe737efea8ff8daa53bbb438e9244ed5"
Jul 15 16:50:38.962044 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:38.962003    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:39.691637 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:39.691601    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:50:40.868349 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:40.868313    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:41.288552 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:41.288461    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:41.370248 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:41.370214    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:41.481563 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:41.481531    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:41.987962 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:41.987924    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:42.091287 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:42.091254    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:42.187969 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:42.187931    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:45.792350 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:45.792312    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:47.154820 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:47.154782    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:47.263010 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:47.262970    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:49.389707 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:49.389672    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:50:53.176192 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:53.176152    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:53.278025 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:53.277984    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:50:53.579717 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:53.579685    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19642093-6jkmk"]
Jul 15 16:50:53.580036 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:53.580023    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02ceffc6-50fd-4ef8-96db-45ff4e0cd4e2" containerName="revoke-keys"
Jul 15 16:50:53.580082 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:53.580037    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="02ceffc6-50fd-4ef8-96db-45ff4e0cd4e2" containerName="revoke-keys"
Jul 15 16:50:53.580125 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:53.580116    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="02ceffc6-50fd-4ef8-96db-45ff4e0cd4e2" containerName="revoke-keys"
Jul 15 16:50:53.582971 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:53.582955    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19642093-6jkmk"
Jul 15 16:50:53.585737 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:53.585709    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lsp65\""
Jul 15 16:50:53.586069 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:53.586050    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 16:50:53.592736 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:53.592716    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19642093-6jkmk"]
Jul 15 16:50:53.676829 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:53.676796    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/574a5817-75e6-4912-a9ab-862438434d15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-19642093-6jkmk\" (UID: \"574a5817-75e6-4912-a9ab-862438434d15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19642093-6jkmk"
Jul 15 16:50:53.778231 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:53.778193    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/574a5817-75e6-4912-a9ab-862438434d15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-19642093-6jkmk\" (UID: \"574a5817-75e6-4912-a9ab-862438434d15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19642093-6jkmk"
Jul 15 16:50:53.778832 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:53.778813    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/574a5817-75e6-4912-a9ab-862438434d15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-19642093-6jkmk\" (UID: \"574a5817-75e6-4912-a9ab-862438434d15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19642093-6jkmk"
Jul 15 16:50:53.893717 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:53.893635    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19642093-6jkmk"
Jul 15 16:50:54.026777 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:54.026702    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19642093-6jkmk"]
Jul 15 16:50:54.029055 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:50:54.029026    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod574a5817_75e6_4912_a9ab_862438434d15.slice/crio-f2f026c02d4162796c4b2921553416696468667565ddcd7e9e04cced6981575c WatchSource:0}: Error finding container f2f026c02d4162796c4b2921553416696468667565ddcd7e9e04cced6981575c: Status 404 returned error can't find the container with id f2f026c02d4162796c4b2921553416696468667565ddcd7e9e04cced6981575c
Jul 15 16:50:54.996871 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:54.996834    2580 generic.go:356] "Generic (PLEG): container finished" podID="574a5817-75e6-4912-a9ab-862438434d15" containerID="4da079e7507ae49470d23fe2e31f17233abae824bb15d97001d338c22bd29afc" exitCode=0
Jul 15 16:50:54.997252 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:54.996912    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19642093-6jkmk" event={"ID":"574a5817-75e6-4912-a9ab-862438434d15","Type":"ContainerDied","Data":"4da079e7507ae49470d23fe2e31f17233abae824bb15d97001d338c22bd29afc"}
Jul 15 16:50:54.997252 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:54.996968    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19642093-6jkmk" event={"ID":"574a5817-75e6-4912-a9ab-862438434d15","Type":"ContainerStarted","Data":"f2f026c02d4162796c4b2921553416696468667565ddcd7e9e04cced6981575c"}
Jul 15 16:50:56.135828 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:56.135797    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19642093-6jkmk"
Jul 15 16:50:56.201999 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:56.201959    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/574a5817-75e6-4912-a9ab-862438434d15-maas-api-service-ca\") pod \"574a5817-75e6-4912-a9ab-862438434d15\" (UID: \"574a5817-75e6-4912-a9ab-862438434d15\") "
Jul 15 16:50:56.202370 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:56.202344    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/574a5817-75e6-4912-a9ab-862438434d15-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "574a5817-75e6-4912-a9ab-862438434d15" (UID: "574a5817-75e6-4912-a9ab-862438434d15"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:50:56.302625 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:56.302595    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/574a5817-75e6-4912-a9ab-862438434d15-maas-api-service-ca\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:50:57.009172 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:57.009146    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19642093-6jkmk"
Jul 15 16:50:57.009337 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:57.009170    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-19642093-6jkmk" event={"ID":"574a5817-75e6-4912-a9ab-862438434d15","Type":"ContainerDied","Data":"f2f026c02d4162796c4b2921553416696468667565ddcd7e9e04cced6981575c"}
Jul 15 16:50:57.009337 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:50:57.009212    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2f026c02d4162796c4b2921553416696468667565ddcd7e9e04cced6981575c"
Jul 15 16:51:04.402288 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:04.402198    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:07.392142 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:07.392103    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:51:08.673222 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:08.673180    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:14.825294 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:14.825256    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:14.925442 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:14.925401    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:19.440240 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:19.440192    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:19.697526 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:19.697435    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:19.843360 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:19.843321    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:19.941009 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:19.940965    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:20.104685 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:20.104643    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:26.152479 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:26.152430    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:26.259012 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:26.258975    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:26.381858 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:26.381822    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:26.576144 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:26.576113    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:32.708544 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:32.708497    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:32.805664 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:32.802587    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:38.333400 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:38.333357    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:38.525504 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:38.525450    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:38.880376 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:38.880349    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-295d8_46356463-bfb1-42b8-8871-71357489a95b/ovn-acl-logging/0.log"
Jul 15 16:51:38.884004 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:38.883980    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-295d8_46356463-bfb1-42b8-8871-71357489a95b/ovn-acl-logging/0.log"
Jul 15 16:51:38.951096 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:38.951063    2580 scope.go:117] "RemoveContainer" containerID="f71997b55cc9e36c8b937131694da30034b261e2bcca7bd5036c3bec3a88840c"
Jul 15 16:51:38.959090 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:38.959072    2580 scope.go:117] "RemoveContainer" containerID="d20b098687e611674d50f55d8b1e0e55870d96673338bfc26770146263a7ffed"
Jul 15 16:51:39.919975 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:39.919943    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:40.397137 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:40.397091    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:40.666143 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:40.666050    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:43.397578 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:43.397529    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:43.523383 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:43.523342    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:43.788115 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:43.788017    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:44.381149 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:44.381114    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:44.680582 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:44.680483    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:44.785892 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:44.785859    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:47.837046 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:47.837008    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:48.191671 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:48.191594    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:51:54.162963 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:54.162899    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:51:57.206432 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:51:57.206398    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:52:17.611163 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:17.611127    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:52:21.240520 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:21.240476    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc2354a8-dxfmz"]
Jul 15 16:52:21.241032 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:21.241010    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="574a5817-75e6-4912-a9ab-862438434d15" containerName="revoke-keys"
Jul 15 16:52:21.241109 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:21.241035    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="574a5817-75e6-4912-a9ab-862438434d15" containerName="revoke-keys"
Jul 15 16:52:21.241161 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:21.241127    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="574a5817-75e6-4912-a9ab-862438434d15" containerName="revoke-keys"
Jul 15 16:52:21.244022 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:21.243998    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc2354a8-dxfmz"
Jul 15 16:52:21.249835 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:21.249817    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lsp65\""
Jul 15 16:52:21.250656 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:21.250638    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 16:52:21.261772 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:21.259483    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc2354a8-dxfmz"]
Jul 15 16:52:21.347038 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:21.346997    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08a89f85-c4ba-4f8d-b669-651314356715-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-dc2354a8-dxfmz\" (UID: \"08a89f85-c4ba-4f8d-b669-651314356715\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc2354a8-dxfmz"
Jul 15 16:52:21.448034 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:21.447991    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08a89f85-c4ba-4f8d-b669-651314356715-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-dc2354a8-dxfmz\" (UID: \"08a89f85-c4ba-4f8d-b669-651314356715\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc2354a8-dxfmz"
Jul 15 16:52:21.448619 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:21.448598    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08a89f85-c4ba-4f8d-b669-651314356715-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-dc2354a8-dxfmz\" (UID: \"08a89f85-c4ba-4f8d-b669-651314356715\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc2354a8-dxfmz"
Jul 15 16:52:21.553384 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:21.553352    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc2354a8-dxfmz"
Jul 15 16:52:21.691410 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:21.691378    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc2354a8-dxfmz"]
Jul 15 16:52:21.694454 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:52:21.694428    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08a89f85_c4ba_4f8d_b669_651314356715.slice/crio-34ab5dd11672b17c4620425547369bf99657b8b3c5814f7c236fadb615a1572c WatchSource:0}: Error finding container 34ab5dd11672b17c4620425547369bf99657b8b3c5814f7c236fadb615a1572c: Status 404 returned error can't find the container with id 34ab5dd11672b17c4620425547369bf99657b8b3c5814f7c236fadb615a1572c
Jul 15 16:52:22.436087 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:22.436058    2580 generic.go:356] "Generic (PLEG): container finished" podID="08a89f85-c4ba-4f8d-b669-651314356715" containerID="1ae668b74a5c9ab3bbf7c35985f6513998a9d23ee5704e5d6e70e29b9d1ace38" exitCode=0
Jul 15 16:52:22.436447 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:22.436116    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc2354a8-dxfmz" event={"ID":"08a89f85-c4ba-4f8d-b669-651314356715","Type":"ContainerDied","Data":"1ae668b74a5c9ab3bbf7c35985f6513998a9d23ee5704e5d6e70e29b9d1ace38"}
Jul 15 16:52:22.436447 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:22.436157    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc2354a8-dxfmz" event={"ID":"08a89f85-c4ba-4f8d-b669-651314356715","Type":"ContainerStarted","Data":"34ab5dd11672b17c4620425547369bf99657b8b3c5814f7c236fadb615a1572c"}
Jul 15 16:52:23.569095 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:23.569069    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc2354a8-dxfmz"
Jul 15 16:52:23.667272 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:23.667233    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08a89f85-c4ba-4f8d-b669-651314356715-maas-api-service-ca\") pod \"08a89f85-c4ba-4f8d-b669-651314356715\" (UID: \"08a89f85-c4ba-4f8d-b669-651314356715\") "
Jul 15 16:52:23.667605 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:23.667583    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08a89f85-c4ba-4f8d-b669-651314356715-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "08a89f85-c4ba-4f8d-b669-651314356715" (UID: "08a89f85-c4ba-4f8d-b669-651314356715"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:52:23.768552 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:23.768466    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08a89f85-c4ba-4f8d-b669-651314356715-maas-api-service-ca\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:52:24.446455 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:24.446428    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc2354a8-dxfmz"
Jul 15 16:52:24.446664 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:24.446461    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dc2354a8-dxfmz" event={"ID":"08a89f85-c4ba-4f8d-b669-651314356715","Type":"ContainerDied","Data":"34ab5dd11672b17c4620425547369bf99657b8b3c5814f7c236fadb615a1572c"}
Jul 15 16:52:24.446664 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:24.446511    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="34ab5dd11672b17c4620425547369bf99657b8b3c5814f7c236fadb615a1572c"
Jul 15 16:52:32.225732 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:32.225631    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:52:35.892988 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:35.892953    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:52:36.431191 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:36.431153    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:52:38.981599 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:38.981570    2580 scope.go:117] "RemoveContainer" containerID="b3142d4aa5cdbefdd921e0cb7d96c263914f79271879f396025b54e04d85086d"
Jul 15 16:52:41.483376 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:52:41.483337    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:53:18.058053 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:53:18.058022    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:53:20.556319 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:53:20.556286    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:53:27.091200 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:53:27.091153    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:53:27.164952 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:53:27.164921    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:53:38.926211 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:53:38.926164    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:53:39.194143 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:53:39.194053    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:53:40.318155 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:53:40.318107    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:53:40.429221 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:53:40.429178    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:53:41.331201 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:53:41.331158    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:53:41.428974 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:53:41.428935    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:53:42.909016 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:53:42.908982    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:53:42.966144 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:53:42.966105    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:53:49.032315 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:53:49.032278    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:53:49.106120 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:53:49.106086    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:53:49.762732 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:53:49.762698    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:53:55.599347 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:53:55.599315    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:53:55.704801 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:53:55.704746    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:53:55.805101 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:53:55.805063    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:53:57.189957 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:53:57.189923    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:54:06.562599 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:06.562506    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:06.803225 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:06.803188    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:08.500253 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:08.500210    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:27.806984 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:27.806947    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:33.813518 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:33.813481    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:33.911610 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:33.911576    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:34.244232 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:34.244139    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:34.576329 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:34.576293    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:34.922552 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:34.922460    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:38.009134 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:38.009100    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:38.118194 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:38.118153    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:38.213976 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:38.213943    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:38.535947 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:38.535901    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:38.787835 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:38.787718    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:38.964000 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:38.963966    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:39.009448 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:39.009422    2580 scope.go:117] "RemoveContainer" containerID="18e295ffbbc9cdcea5097b484b1cb573b145b131fbe4bb6d80f43bd85f52f985"
Jul 15 16:54:42.632736 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:42.632697    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:42.872564 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:42.872524    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:43.200508 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:43.200472    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:43.725959 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:43.725918    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:45.188321 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:45.188285    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:54:46.501364 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:46.501325    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:46.593271 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:46.593237    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q"]
Jul 15 16:54:46.593616 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:46.593602    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08a89f85-c4ba-4f8d-b669-651314356715" containerName="revoke-keys"
Jul 15 16:54:46.593661 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:46.593618    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="08a89f85-c4ba-4f8d-b669-651314356715" containerName="revoke-keys"
Jul 15 16:54:46.593695 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:46.593682    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="08a89f85-c4ba-4f8d-b669-651314356715" containerName="revoke-keys"
Jul 15 16:54:46.595564 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:46.595533    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:46.597006 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:46.596988    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q"
Jul 15 16:54:46.599567 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:46.599543    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-cd9127-serving-cert\""
Jul 15 16:54:46.599567 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:46.599561    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-v8jb8\""
Jul 15 16:54:46.599882 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:46.599869    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 16:54:46.600532 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:46.600515    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 16:54:46.602272 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:46.602251    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e02f4cd-627f-4778-ba5e-f5b31b933518-maas-api-tls\") pod \"maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q\" (UID: \"3e02f4cd-627f-4778-ba5e-f5b31b933518\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q"
Jul 15 16:54:46.602386 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:46.602335    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twpp7\" (UniqueName: \"kubernetes.io/projected/3e02f4cd-627f-4778-ba5e-f5b31b933518-kube-api-access-twpp7\") pod \"maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q\" (UID: \"3e02f4cd-627f-4778-ba5e-f5b31b933518\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q"
Jul 15 16:54:46.602386 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:46.602369    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e02f4cd-627f-4778-ba5e-f5b31b933518-openshift-service-ca\") pod \"maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q\" (UID: \"3e02f4cd-627f-4778-ba5e-f5b31b933518\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q"
Jul 15 16:54:46.607777 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:46.607736    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q"]
Jul 15 16:54:46.670727 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:46.670695    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:46.703448 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:46.703413    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twpp7\" (UniqueName: \"kubernetes.io/projected/3e02f4cd-627f-4778-ba5e-f5b31b933518-kube-api-access-twpp7\") pod \"maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q\" (UID: \"3e02f4cd-627f-4778-ba5e-f5b31b933518\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q"
Jul 15 16:54:46.703616 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:46.703451    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e02f4cd-627f-4778-ba5e-f5b31b933518-openshift-service-ca\") pod \"maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q\" (UID: \"3e02f4cd-627f-4778-ba5e-f5b31b933518\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q"
Jul 15 16:54:46.703616 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:46.703523    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e02f4cd-627f-4778-ba5e-f5b31b933518-maas-api-tls\") pod \"maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q\" (UID: \"3e02f4cd-627f-4778-ba5e-f5b31b933518\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q"
Jul 15 16:54:46.703700 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:54:46.703615    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-serving-cert: secret "maas-api-e2e-inf-b-cd9127-serving-cert" not found
Jul 15 16:54:46.703738 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:54:46.703699    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e02f4cd-627f-4778-ba5e-f5b31b933518-maas-api-tls podName:3e02f4cd-627f-4778-ba5e-f5b31b933518 nodeName:}" failed. No retries permitted until 2026-07-15 16:54:47.203677987 +0000 UTC m=+2288.850718504 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3e02f4cd-627f-4778-ba5e-f5b31b933518-maas-api-tls") pod "maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q" (UID: "3e02f4cd-627f-4778-ba5e-f5b31b933518") : secret "maas-api-e2e-inf-b-cd9127-serving-cert" not found
Jul 15 16:54:46.704246 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:46.704226    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e02f4cd-627f-4778-ba5e-f5b31b933518-openshift-service-ca\") pod \"maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q\" (UID: \"3e02f4cd-627f-4778-ba5e-f5b31b933518\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q"
Jul 15 16:54:46.719195 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:46.719167    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twpp7\" (UniqueName: \"kubernetes.io/projected/3e02f4cd-627f-4778-ba5e-f5b31b933518-kube-api-access-twpp7\") pod \"maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q\" (UID: \"3e02f4cd-627f-4778-ba5e-f5b31b933518\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q"
Jul 15 16:54:47.208215 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:47.208175    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e02f4cd-627f-4778-ba5e-f5b31b933518-maas-api-tls\") pod \"maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q\" (UID: \"3e02f4cd-627f-4778-ba5e-f5b31b933518\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q"
Jul 15 16:54:47.208441 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:54:47.208349    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-serving-cert: secret "maas-api-e2e-inf-b-cd9127-serving-cert" not found
Jul 15 16:54:47.208514 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:54:47.208442    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e02f4cd-627f-4778-ba5e-f5b31b933518-maas-api-tls podName:3e02f4cd-627f-4778-ba5e-f5b31b933518 nodeName:}" failed. No retries permitted until 2026-07-15 16:54:48.208410678 +0000 UTC m=+2289.855451206 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3e02f4cd-627f-4778-ba5e-f5b31b933518-maas-api-tls") pod "maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q" (UID: "3e02f4cd-627f-4778-ba5e-f5b31b933518") : secret "maas-api-e2e-inf-b-cd9127-serving-cert" not found
Jul 15 16:54:47.465500 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:47.465408    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:47.566898 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:47.566856    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:48.216007 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:48.215968    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e02f4cd-627f-4778-ba5e-f5b31b933518-maas-api-tls\") pod \"maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q\" (UID: \"3e02f4cd-627f-4778-ba5e-f5b31b933518\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q"
Jul 15 16:54:48.218375 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:48.218346    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e02f4cd-627f-4778-ba5e-f5b31b933518-maas-api-tls\") pod \"maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q\" (UID: \"3e02f4cd-627f-4778-ba5e-f5b31b933518\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q"
Jul 15 16:54:48.407940 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:48.407902    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q"
Jul 15 16:54:48.538421 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:48.538394    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q"]
Jul 15 16:54:48.540278 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:54:48.540251    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e02f4cd_627f_4778_ba5e_f5b31b933518.slice/crio-595d101ae65904d27b3874de427932260b34947dca431013a63f56ad160e4ac2 WatchSource:0}: Error finding container 595d101ae65904d27b3874de427932260b34947dca431013a63f56ad160e4ac2: Status 404 returned error can't find the container with id 595d101ae65904d27b3874de427932260b34947dca431013a63f56ad160e4ac2
Jul 15 16:54:48.541972 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:48.541952    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 16:54:49.139658 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:49.139619    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q" event={"ID":"3e02f4cd-627f-4778-ba5e-f5b31b933518","Type":"ContainerStarted","Data":"595d101ae65904d27b3874de427932260b34947dca431013a63f56ad160e4ac2"}
Jul 15 16:54:50.991699 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:50.991561    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:51.151504 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:51.151471    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q" event={"ID":"3e02f4cd-627f-4778-ba5e-f5b31b933518","Type":"ContainerStarted","Data":"f2797904333c8ef9f6b7c3c5317545cff78a06270a3a4b33a2a8e2c08530e6d1"}
Jul 15 16:54:51.151681 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:51.151547    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q"
Jul 15 16:54:51.178358 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:51.178304    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q" podStartSLOduration=3.085001054 podStartE2EDuration="5.178288162s" totalImagesPullingTime="2.093287108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:54:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:54:51.176942343 +0000 UTC m=+2292.823982909" watchObservedRunningTime="2026-07-15 16:54:51.178288162 +0000 UTC m=+2292.825328697"
Jul 15 16:54:55.627593 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.627552    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"]
Jul 15 16:54:55.631473 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.631451    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:54:55.635223 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.635197    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-5b944f\"/\"openshift-service-ca.crt\""
Jul 15 16:54:55.635341 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.635258    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-5b944f\"/\"kube-root-ca.crt\""
Jul 15 16:54:55.635341 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.635276    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-5b944f\"/\"test-model-5b944f-kserve-self-signed-certs\""
Jul 15 16:54:55.635341 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.635295    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-5b944f\"/\"default-dockercfg-lj2hs\""
Jul 15 16:54:55.641528 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.641500    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"]
Jul 15 16:54:55.655956 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.655919    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:55.671845 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.671818    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcbd8363-e420-40ef-947d-b06cff58b410-dshm\") pod \"test-model-5b944f-kserve-8ccdc9765-hbxpt\" (UID: \"fcbd8363-e420-40ef-947d-b06cff58b410\") " pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:54:55.672019 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.671864    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcbd8363-e420-40ef-947d-b06cff58b410-model-cache\") pod \"test-model-5b944f-kserve-8ccdc9765-hbxpt\" (UID: \"fcbd8363-e420-40ef-947d-b06cff58b410\") " pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:54:55.672019 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.671898    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcbd8363-e420-40ef-947d-b06cff58b410-tls-certs\") pod \"test-model-5b944f-kserve-8ccdc9765-hbxpt\" (UID: \"fcbd8363-e420-40ef-947d-b06cff58b410\") " pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:54:55.672019 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.671941    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ptv9\" (UniqueName: \"kubernetes.io/projected/fcbd8363-e420-40ef-947d-b06cff58b410-kube-api-access-2ptv9\") pod \"test-model-5b944f-kserve-8ccdc9765-hbxpt\" (UID: \"fcbd8363-e420-40ef-947d-b06cff58b410\") " pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:54:55.672181 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.672027    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcbd8363-e420-40ef-947d-b06cff58b410-home\") pod \"test-model-5b944f-kserve-8ccdc9765-hbxpt\" (UID: \"fcbd8363-e420-40ef-947d-b06cff58b410\") " pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:54:55.672181 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.672104    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcbd8363-e420-40ef-947d-b06cff58b410-tmp-dir\") pod \"test-model-5b944f-kserve-8ccdc9765-hbxpt\" (UID: \"fcbd8363-e420-40ef-947d-b06cff58b410\") " pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:54:55.772531 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.772487    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcbd8363-e420-40ef-947d-b06cff58b410-tmp-dir\") pod \"test-model-5b944f-kserve-8ccdc9765-hbxpt\" (UID: \"fcbd8363-e420-40ef-947d-b06cff58b410\") " pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:54:55.772726 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.772537    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcbd8363-e420-40ef-947d-b06cff58b410-dshm\") pod \"test-model-5b944f-kserve-8ccdc9765-hbxpt\" (UID: \"fcbd8363-e420-40ef-947d-b06cff58b410\") " pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:54:55.772726 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.772569    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcbd8363-e420-40ef-947d-b06cff58b410-model-cache\") pod \"test-model-5b944f-kserve-8ccdc9765-hbxpt\" (UID: \"fcbd8363-e420-40ef-947d-b06cff58b410\") " pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:54:55.772726 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.772585    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcbd8363-e420-40ef-947d-b06cff58b410-tls-certs\") pod \"test-model-5b944f-kserve-8ccdc9765-hbxpt\" (UID: \"fcbd8363-e420-40ef-947d-b06cff58b410\") " pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:54:55.772726 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.772618    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ptv9\" (UniqueName: \"kubernetes.io/projected/fcbd8363-e420-40ef-947d-b06cff58b410-kube-api-access-2ptv9\") pod \"test-model-5b944f-kserve-8ccdc9765-hbxpt\" (UID: \"fcbd8363-e420-40ef-947d-b06cff58b410\") " pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:54:55.772726 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.772683    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcbd8363-e420-40ef-947d-b06cff58b410-home\") pod \"test-model-5b944f-kserve-8ccdc9765-hbxpt\" (UID: \"fcbd8363-e420-40ef-947d-b06cff58b410\") " pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:54:55.773035 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.772981    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcbd8363-e420-40ef-947d-b06cff58b410-tmp-dir\") pod \"test-model-5b944f-kserve-8ccdc9765-hbxpt\" (UID: \"fcbd8363-e420-40ef-947d-b06cff58b410\") " pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:54:55.773093 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.773048    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcbd8363-e420-40ef-947d-b06cff58b410-model-cache\") pod \"test-model-5b944f-kserve-8ccdc9765-hbxpt\" (UID: \"fcbd8363-e420-40ef-947d-b06cff58b410\") " pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:54:55.773142 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.773095    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcbd8363-e420-40ef-947d-b06cff58b410-home\") pod \"test-model-5b944f-kserve-8ccdc9765-hbxpt\" (UID: \"fcbd8363-e420-40ef-947d-b06cff58b410\") " pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:54:55.774860 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.774834    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcbd8363-e420-40ef-947d-b06cff58b410-dshm\") pod \"test-model-5b944f-kserve-8ccdc9765-hbxpt\" (UID: \"fcbd8363-e420-40ef-947d-b06cff58b410\") " pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:54:55.775131 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.775114    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcbd8363-e420-40ef-947d-b06cff58b410-tls-certs\") pod \"test-model-5b944f-kserve-8ccdc9765-hbxpt\" (UID: \"fcbd8363-e420-40ef-947d-b06cff58b410\") " pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:54:55.782167 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.782144    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ptv9\" (UniqueName: \"kubernetes.io/projected/fcbd8363-e420-40ef-947d-b06cff58b410-kube-api-access-2ptv9\") pod \"test-model-5b944f-kserve-8ccdc9765-hbxpt\" (UID: \"fcbd8363-e420-40ef-947d-b06cff58b410\") " pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:54:55.942463 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:55.942364    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:54:56.073103 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:56.073073    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"]
Jul 15 16:54:56.075245 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:54:56.075213    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfcbd8363_e420_40ef_947d_b06cff58b410.slice/crio-fe43f4ac82c55693463755caf15feb6e66714e722e85dd40257cd5b29d0605de WatchSource:0}: Error finding container fe43f4ac82c55693463755caf15feb6e66714e722e85dd40257cd5b29d0605de: Status 404 returned error can't find the container with id fe43f4ac82c55693463755caf15feb6e66714e722e85dd40257cd5b29d0605de
Jul 15 16:54:56.177633 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:56.177594    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt" event={"ID":"fcbd8363-e420-40ef-947d-b06cff58b410","Type":"ContainerStarted","Data":"6e3bb09bf10e79294f65765e883dc2469a86ddbefe5b9f4f08dfcda7b66b7ba1"}
Jul 15 16:54:56.177633 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:56.177633    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt" event={"ID":"fcbd8363-e420-40ef-947d-b06cff58b410","Type":"ContainerStarted","Data":"fe43f4ac82c55693463755caf15feb6e66714e722e85dd40257cd5b29d0605de"}
Jul 15 16:54:56.203226 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:56.200077    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt" podStartSLOduration=1.20005761 podStartE2EDuration="1.20005761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:54:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:54:56.195395885 +0000 UTC m=+2297.842436423" watchObservedRunningTime="2026-07-15 16:54:56.20005761 +0000 UTC m=+2297.847098149"
Jul 15 16:54:57.163212 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:57.163173    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q"
Jul 15 16:54:58.093914 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:58.093878    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:54:59.698058 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:59.698015    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:54:59.797571 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:54:59.797531    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:55:05.942543 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:05.942500    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:55:05.943000 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:05.942561    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:55:05.955047 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:05.955010    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:55:06.240867 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:06.240788    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:55:08.381301 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:08.381264    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:55:08.482799 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:08.482750    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:55:11.796373 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:11.796333    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:55:15.656361 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:15.656318    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:55:23.079361 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:23.079326    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:55:25.673352 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:25.673312    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:55:40.409045 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:40.408963    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"]
Jul 15 16:55:40.509684 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:40.509590    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:55:41.345860 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:41.345816    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:55:41.447910 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:41.447866    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5b944f-r48cp"]
Jul 15 16:55:41.455082 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:41.455057    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5b944f-r48cp"
Jul 15 16:55:41.457792 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:41.457740    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lsp65\""
Jul 15 16:55:41.460307 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:41.460284    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5b944f-r48cp"]
Jul 15 16:55:41.554171 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:41.554130    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a5a903b-dcd4-4321-9106-08dec8627cf0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-5b944f-r48cp\" (UID: \"9a5a903b-dcd4-4321-9106-08dec8627cf0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5b944f-r48cp"
Jul 15 16:55:41.655382 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:41.655289    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a5a903b-dcd4-4321-9106-08dec8627cf0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-5b944f-r48cp\" (UID: \"9a5a903b-dcd4-4321-9106-08dec8627cf0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5b944f-r48cp"
Jul 15 16:55:41.656017 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:41.655996    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a5a903b-dcd4-4321-9106-08dec8627cf0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-5b944f-r48cp\" (UID: \"9a5a903b-dcd4-4321-9106-08dec8627cf0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5b944f-r48cp"
Jul 15 16:55:41.767200 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:41.767177    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5b944f-r48cp"
Jul 15 16:55:41.900165 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:41.900137    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5b944f-r48cp"]
Jul 15 16:55:41.902209 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:55:41.902182    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a5a903b_dcd4_4321_9106_08dec8627cf0.slice/crio-d6b17a393f2c8f770778996004eb2f9e48db565d84265646f39bf7e1a8e15c39 WatchSource:0}: Error finding container d6b17a393f2c8f770778996004eb2f9e48db565d84265646f39bf7e1a8e15c39: Status 404 returned error can't find the container with id d6b17a393f2c8f770778996004eb2f9e48db565d84265646f39bf7e1a8e15c39
Jul 15 16:55:42.116180 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:42.116143    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:55:42.215683 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:42.215590    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:55:42.403388 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:42.403350    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5b944f-r48cp" event={"ID":"9a5a903b-dcd4-4321-9106-08dec8627cf0","Type":"ContainerStarted","Data":"4523123f5875b00e2da8bacc2726fa4450163ce5083ab6dadbb9cd417cad63c1"}
Jul 15 16:55:42.403561 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:42.403403    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5b944f-r48cp" event={"ID":"9a5a903b-dcd4-4321-9106-08dec8627cf0","Type":"ContainerStarted","Data":"d6b17a393f2c8f770778996004eb2f9e48db565d84265646f39bf7e1a8e15c39"}
Jul 15 16:55:42.419868 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:42.419809    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5b944f-r48cp" podStartSLOduration=1.419793969 podStartE2EDuration="1.419793969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:55:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:55:42.418027333 +0000 UTC m=+2344.065067870" watchObservedRunningTime="2026-07-15 16:55:42.419793969 +0000 UTC m=+2344.066834505"
Jul 15 16:55:43.409919 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:43.409884    2580 generic.go:356] "Generic (PLEG): container finished" podID="9a5a903b-dcd4-4321-9106-08dec8627cf0" containerID="4523123f5875b00e2da8bacc2726fa4450163ce5083ab6dadbb9cd417cad63c1" exitCode=0
Jul 15 16:55:43.410308 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:43.409972    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5b944f-r48cp" event={"ID":"9a5a903b-dcd4-4321-9106-08dec8627cf0","Type":"ContainerDied","Data":"4523123f5875b00e2da8bacc2726fa4450163ce5083ab6dadbb9cd417cad63c1"}
Jul 15 16:55:44.546654 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:44.546627    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5b944f-r48cp"
Jul 15 16:55:44.682223 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:44.682133    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a5a903b-dcd4-4321-9106-08dec8627cf0-maas-api-service-ca\") pod \"9a5a903b-dcd4-4321-9106-08dec8627cf0\" (UID: \"9a5a903b-dcd4-4321-9106-08dec8627cf0\") "
Jul 15 16:55:44.682501 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:44.682474    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a5a903b-dcd4-4321-9106-08dec8627cf0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9a5a903b-dcd4-4321-9106-08dec8627cf0" (UID: "9a5a903b-dcd4-4321-9106-08dec8627cf0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:55:44.783525 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:44.783485    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a5a903b-dcd4-4321-9106-08dec8627cf0-maas-api-service-ca\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:55:45.420481 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:45.420447    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5b944f-r48cp" event={"ID":"9a5a903b-dcd4-4321-9106-08dec8627cf0","Type":"ContainerDied","Data":"d6b17a393f2c8f770778996004eb2f9e48db565d84265646f39bf7e1a8e15c39"}
Jul 15 16:55:45.420481 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:45.420469    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5b944f-r48cp"
Jul 15 16:55:45.420734 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:45.420498    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d6b17a393f2c8f770778996004eb2f9e48db565d84265646f39bf7e1a8e15c39"
Jul 15 16:55:46.989709 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:46.989665    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:55:51.952012 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:51.951975    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:55:55.432801 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:55.432738    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt" podUID="fcbd8363-e420-40ef-947d-b06cff58b410" containerName="main" containerID="cri-o://6e3bb09bf10e79294f65765e883dc2469a86ddbefe5b9f4f08dfcda7b66b7ba1" gracePeriod=45
Jul 15 16:55:55.686874 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:55.686812    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:55:55.872807 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:55.872744    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcbd8363-e420-40ef-947d-b06cff58b410-home\") pod \"fcbd8363-e420-40ef-947d-b06cff58b410\" (UID: \"fcbd8363-e420-40ef-947d-b06cff58b410\") "
Jul 15 16:55:55.872981 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:55.872818    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcbd8363-e420-40ef-947d-b06cff58b410-dshm\") pod \"fcbd8363-e420-40ef-947d-b06cff58b410\" (UID: \"fcbd8363-e420-40ef-947d-b06cff58b410\") "
Jul 15 16:55:55.872981 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:55.872840    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcbd8363-e420-40ef-947d-b06cff58b410-model-cache\") pod \"fcbd8363-e420-40ef-947d-b06cff58b410\" (UID: \"fcbd8363-e420-40ef-947d-b06cff58b410\") "
Jul 15 16:55:55.872981 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:55.872865    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ptv9\" (UniqueName: \"kubernetes.io/projected/fcbd8363-e420-40ef-947d-b06cff58b410-kube-api-access-2ptv9\") pod \"fcbd8363-e420-40ef-947d-b06cff58b410\" (UID: \"fcbd8363-e420-40ef-947d-b06cff58b410\") "
Jul 15 16:55:55.872981 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:55.872894    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcbd8363-e420-40ef-947d-b06cff58b410-tmp-dir\") pod \"fcbd8363-e420-40ef-947d-b06cff58b410\" (UID: \"fcbd8363-e420-40ef-947d-b06cff58b410\") "
Jul 15 16:55:55.872981 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:55.872931    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcbd8363-e420-40ef-947d-b06cff58b410-tls-certs\") pod \"fcbd8363-e420-40ef-947d-b06cff58b410\" (UID: \"fcbd8363-e420-40ef-947d-b06cff58b410\") "
Jul 15 16:55:55.873249 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:55.873055    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcbd8363-e420-40ef-947d-b06cff58b410-home" (OuterVolumeSpecName: "home") pod "fcbd8363-e420-40ef-947d-b06cff58b410" (UID: "fcbd8363-e420-40ef-947d-b06cff58b410"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:55:55.873249 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:55.873159    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcbd8363-e420-40ef-947d-b06cff58b410-model-cache" (OuterVolumeSpecName: "model-cache") pod "fcbd8363-e420-40ef-947d-b06cff58b410" (UID: "fcbd8363-e420-40ef-947d-b06cff58b410"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:55:55.873365 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:55.873275    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcbd8363-e420-40ef-947d-b06cff58b410-home\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:55:55.873365 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:55.873271    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcbd8363-e420-40ef-947d-b06cff58b410-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fcbd8363-e420-40ef-947d-b06cff58b410" (UID: "fcbd8363-e420-40ef-947d-b06cff58b410"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:55:55.873365 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:55.873292    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcbd8363-e420-40ef-947d-b06cff58b410-model-cache\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:55:55.875168 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:55.875130    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcbd8363-e420-40ef-947d-b06cff58b410-kube-api-access-2ptv9" (OuterVolumeSpecName: "kube-api-access-2ptv9") pod "fcbd8363-e420-40ef-947d-b06cff58b410" (UID: "fcbd8363-e420-40ef-947d-b06cff58b410"). InnerVolumeSpecName "kube-api-access-2ptv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:55:55.875168 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:55.875139    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcbd8363-e420-40ef-947d-b06cff58b410-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fcbd8363-e420-40ef-947d-b06cff58b410" (UID: "fcbd8363-e420-40ef-947d-b06cff58b410"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:55:55.875340 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:55.875223    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcbd8363-e420-40ef-947d-b06cff58b410-dshm" (OuterVolumeSpecName: "dshm") pod "fcbd8363-e420-40ef-947d-b06cff58b410" (UID: "fcbd8363-e420-40ef-947d-b06cff58b410"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 16:55:55.974673 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:55.974578    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcbd8363-e420-40ef-947d-b06cff58b410-dshm\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:55:55.974673 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:55.974611    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ptv9\" (UniqueName: \"kubernetes.io/projected/fcbd8363-e420-40ef-947d-b06cff58b410-kube-api-access-2ptv9\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:55:55.974673 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:55.974622    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcbd8363-e420-40ef-947d-b06cff58b410-tmp-dir\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:55:55.974673 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:55.974631    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcbd8363-e420-40ef-947d-b06cff58b410-tls-certs\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:55:56.475509 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:56.475475    2580 generic.go:356] "Generic (PLEG): container finished" podID="fcbd8363-e420-40ef-947d-b06cff58b410" containerID="6e3bb09bf10e79294f65765e883dc2469a86ddbefe5b9f4f08dfcda7b66b7ba1" exitCode=0
Jul 15 16:55:56.475963 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:56.475519    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"
Jul 15 16:55:56.475963 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:56.475532    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt" event={"ID":"fcbd8363-e420-40ef-947d-b06cff58b410","Type":"ContainerDied","Data":"6e3bb09bf10e79294f65765e883dc2469a86ddbefe5b9f4f08dfcda7b66b7ba1"}
Jul 15 16:55:56.475963 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:56.475564    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt" event={"ID":"fcbd8363-e420-40ef-947d-b06cff58b410","Type":"ContainerDied","Data":"fe43f4ac82c55693463755caf15feb6e66714e722e85dd40257cd5b29d0605de"}
Jul 15 16:55:56.475963 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:56.475583    2580 scope.go:117] "RemoveContainer" containerID="6e3bb09bf10e79294f65765e883dc2469a86ddbefe5b9f4f08dfcda7b66b7ba1"
Jul 15 16:55:56.485012 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:56.484994    2580 scope.go:117] "RemoveContainer" containerID="6e3bb09bf10e79294f65765e883dc2469a86ddbefe5b9f4f08dfcda7b66b7ba1"
Jul 15 16:55:56.485259 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:55:56.485241    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e3bb09bf10e79294f65765e883dc2469a86ddbefe5b9f4f08dfcda7b66b7ba1\": container with ID starting with 6e3bb09bf10e79294f65765e883dc2469a86ddbefe5b9f4f08dfcda7b66b7ba1 not found: ID does not exist" containerID="6e3bb09bf10e79294f65765e883dc2469a86ddbefe5b9f4f08dfcda7b66b7ba1"
Jul 15 16:55:56.485302 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:56.485269    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e3bb09bf10e79294f65765e883dc2469a86ddbefe5b9f4f08dfcda7b66b7ba1"} err="failed to get container status \"6e3bb09bf10e79294f65765e883dc2469a86ddbefe5b9f4f08dfcda7b66b7ba1\": rpc error: code = NotFound desc = could not find container \"6e3bb09bf10e79294f65765e883dc2469a86ddbefe5b9f4f08dfcda7b66b7ba1\": container with ID starting with 6e3bb09bf10e79294f65765e883dc2469a86ddbefe5b9f4f08dfcda7b66b7ba1 not found: ID does not exist"
Jul 15 16:55:56.500008 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:56.499979    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"]
Jul 15 16:55:56.506956 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:56.506933    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-5b944f/test-model-5b944f-kserve-8ccdc9765-hbxpt"]
Jul 15 16:55:56.553520 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:56.553484    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:55:56.878711 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:56.878680    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcbd8363-e420-40ef-947d-b06cff58b410" path="/var/lib/kubelet/pods/fcbd8363-e420-40ef-947d-b06cff58b410/volumes"
Jul 15 16:55:58.686586 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:55:58.686554    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:56:02.421165 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:02.421128    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:02.483206 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:02.483172    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:07.312450 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:07.312415    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:07.563546 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:07.563449    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:07.695879 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:07.695848    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cd9127-lk7c2"]
Jul 15 16:56:07.696238 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:07.696225    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcbd8363-e420-40ef-947d-b06cff58b410" containerName="main"
Jul 15 16:56:07.696284 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:07.696240    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcbd8363-e420-40ef-947d-b06cff58b410" containerName="main"
Jul 15 16:56:07.696284 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:07.696251    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a5a903b-dcd4-4321-9106-08dec8627cf0" containerName="revoke-keys"
Jul 15 16:56:07.696284 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:07.696257    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a5a903b-dcd4-4321-9106-08dec8627cf0" containerName="revoke-keys"
Jul 15 16:56:07.696382 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:07.696307    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a5a903b-dcd4-4321-9106-08dec8627cf0" containerName="revoke-keys"
Jul 15 16:56:07.696382 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:07.696316    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcbd8363-e420-40ef-947d-b06cff58b410" containerName="main"
Jul 15 16:56:07.701148 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:07.701132    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cd9127-lk7c2"
Jul 15 16:56:07.703908 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:07.703889    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lsp65\""
Jul 15 16:56:07.709007 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:07.708982    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cd9127-lk7c2"]
Jul 15 16:56:07.770214 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:07.770166    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecfc41f1-5028-491f-8d95-cd3a2564b841-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-cd9127-lk7c2\" (UID: \"ecfc41f1-5028-491f-8d95-cd3a2564b841\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cd9127-lk7c2"
Jul 15 16:56:07.870945 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:07.870853    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecfc41f1-5028-491f-8d95-cd3a2564b841-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-cd9127-lk7c2\" (UID: \"ecfc41f1-5028-491f-8d95-cd3a2564b841\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cd9127-lk7c2"
Jul 15 16:56:07.871508 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:07.871488    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecfc41f1-5028-491f-8d95-cd3a2564b841-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-cd9127-lk7c2\" (UID: \"ecfc41f1-5028-491f-8d95-cd3a2564b841\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cd9127-lk7c2"
Jul 15 16:56:08.012603 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:08.012563    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cd9127-lk7c2"
Jul 15 16:56:08.157448 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:08.157360    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cd9127-lk7c2"]
Jul 15 16:56:08.160422 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:56:08.160393    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecfc41f1_5028_491f_8d95_cd3a2564b841.slice/crio-5491d9257b29e056742cd95b1546ba1149b1c1d21a2c86a111f55125ce854279 WatchSource:0}: Error finding container 5491d9257b29e056742cd95b1546ba1149b1c1d21a2c86a111f55125ce854279: Status 404 returned error can't find the container with id 5491d9257b29e056742cd95b1546ba1149b1c1d21a2c86a111f55125ce854279
Jul 15 16:56:08.416553 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:08.416463    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:08.518270 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:08.518231    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:08.544869 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:08.544836    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cd9127-lk7c2" event={"ID":"ecfc41f1-5028-491f-8d95-cd3a2564b841","Type":"ContainerStarted","Data":"7475d0b3cc72f4e5e8fbbeb37ee8ce7d90b4944d40f725979067c7d599db34c5"}
Jul 15 16:56:08.545062 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:08.544888    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cd9127-lk7c2" event={"ID":"ecfc41f1-5028-491f-8d95-cd3a2564b841","Type":"ContainerStarted","Data":"5491d9257b29e056742cd95b1546ba1149b1c1d21a2c86a111f55125ce854279"}
Jul 15 16:56:08.569877 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:08.569806    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cd9127-lk7c2" podStartSLOduration=1.5697851649999999 podStartE2EDuration="1.569785165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:56:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:08.569558644 +0000 UTC m=+2370.216599180" watchObservedRunningTime="2026-07-15 16:56:08.569785165 +0000 UTC m=+2370.216825701"
Jul 15 16:56:09.551639 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:09.551597    2580 generic.go:356] "Generic (PLEG): container finished" podID="ecfc41f1-5028-491f-8d95-cd3a2564b841" containerID="7475d0b3cc72f4e5e8fbbeb37ee8ce7d90b4944d40f725979067c7d599db34c5" exitCode=0
Jul 15 16:56:09.552070 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:09.551682    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cd9127-lk7c2" event={"ID":"ecfc41f1-5028-491f-8d95-cd3a2564b841","Type":"ContainerDied","Data":"7475d0b3cc72f4e5e8fbbeb37ee8ce7d90b4944d40f725979067c7d599db34c5"}
Jul 15 16:56:10.689100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:10.689075    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cd9127-lk7c2"
Jul 15 16:56:10.693157 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:10.693135    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecfc41f1-5028-491f-8d95-cd3a2564b841-maas-api-service-ca\") pod \"ecfc41f1-5028-491f-8d95-cd3a2564b841\" (UID: \"ecfc41f1-5028-491f-8d95-cd3a2564b841\") "
Jul 15 16:56:10.693468 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:10.693445    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecfc41f1-5028-491f-8d95-cd3a2564b841-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ecfc41f1-5028-491f-8d95-cd3a2564b841" (UID: "ecfc41f1-5028-491f-8d95-cd3a2564b841"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:56:10.794639 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:10.794608    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecfc41f1-5028-491f-8d95-cd3a2564b841-maas-api-service-ca\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:56:11.563233 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:11.563205    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cd9127-lk7c2"
Jul 15 16:56:11.563494 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:11.563245    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-cd9127-lk7c2" event={"ID":"ecfc41f1-5028-491f-8d95-cd3a2564b841","Type":"ContainerDied","Data":"5491d9257b29e056742cd95b1546ba1149b1c1d21a2c86a111f55125ce854279"}
Jul 15 16:56:11.563494 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:11.563294    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5491d9257b29e056742cd95b1546ba1149b1c1d21a2c86a111f55125ce854279"
Jul 15 16:56:18.199280 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:18.199247    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q"]
Jul 15 16:56:18.199687 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:18.199458    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q" podUID="3e02f4cd-627f-4778-ba5e-f5b31b933518" containerName="maas-api" containerID="cri-o://f2797904333c8ef9f6b7c3c5317545cff78a06270a3a4b33a2a8e2c08530e6d1" gracePeriod=30
Jul 15 16:56:18.228302 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:18.228272    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:18.448575 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:18.448549    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q"
Jul 15 16:56:18.564536 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:18.564499    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twpp7\" (UniqueName: \"kubernetes.io/projected/3e02f4cd-627f-4778-ba5e-f5b31b933518-kube-api-access-twpp7\") pod \"3e02f4cd-627f-4778-ba5e-f5b31b933518\" (UID: \"3e02f4cd-627f-4778-ba5e-f5b31b933518\") "
Jul 15 16:56:18.564721 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:18.564580    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e02f4cd-627f-4778-ba5e-f5b31b933518-openshift-service-ca\") pod \"3e02f4cd-627f-4778-ba5e-f5b31b933518\" (UID: \"3e02f4cd-627f-4778-ba5e-f5b31b933518\") "
Jul 15 16:56:18.564721 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:18.564606    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e02f4cd-627f-4778-ba5e-f5b31b933518-maas-api-tls\") pod \"3e02f4cd-627f-4778-ba5e-f5b31b933518\" (UID: \"3e02f4cd-627f-4778-ba5e-f5b31b933518\") "
Jul 15 16:56:18.564995 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:18.564964    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e02f4cd-627f-4778-ba5e-f5b31b933518-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3e02f4cd-627f-4778-ba5e-f5b31b933518" (UID: "3e02f4cd-627f-4778-ba5e-f5b31b933518"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:56:18.566785 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:18.566743    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e02f4cd-627f-4778-ba5e-f5b31b933518-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3e02f4cd-627f-4778-ba5e-f5b31b933518" (UID: "3e02f4cd-627f-4778-ba5e-f5b31b933518"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:56:18.566876 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:18.566819    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e02f4cd-627f-4778-ba5e-f5b31b933518-kube-api-access-twpp7" (OuterVolumeSpecName: "kube-api-access-twpp7") pod "3e02f4cd-627f-4778-ba5e-f5b31b933518" (UID: "3e02f4cd-627f-4778-ba5e-f5b31b933518"). InnerVolumeSpecName "kube-api-access-twpp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:56:18.600396 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:18.600365    2580 generic.go:356] "Generic (PLEG): container finished" podID="3e02f4cd-627f-4778-ba5e-f5b31b933518" containerID="f2797904333c8ef9f6b7c3c5317545cff78a06270a3a4b33a2a8e2c08530e6d1" exitCode=0
Jul 15 16:56:18.600492 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:18.600403    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q" event={"ID":"3e02f4cd-627f-4778-ba5e-f5b31b933518","Type":"ContainerDied","Data":"f2797904333c8ef9f6b7c3c5317545cff78a06270a3a4b33a2a8e2c08530e6d1"}
Jul 15 16:56:18.600492 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:18.600416    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q"
Jul 15 16:56:18.600492 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:18.600448    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q" event={"ID":"3e02f4cd-627f-4778-ba5e-f5b31b933518","Type":"ContainerDied","Data":"595d101ae65904d27b3874de427932260b34947dca431013a63f56ad160e4ac2"}
Jul 15 16:56:18.600492 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:18.600470    2580 scope.go:117] "RemoveContainer" containerID="f2797904333c8ef9f6b7c3c5317545cff78a06270a3a4b33a2a8e2c08530e6d1"
Jul 15 16:56:18.609670 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:18.609649    2580 scope.go:117] "RemoveContainer" containerID="f2797904333c8ef9f6b7c3c5317545cff78a06270a3a4b33a2a8e2c08530e6d1"
Jul 15 16:56:18.609948 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:56:18.609927    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2797904333c8ef9f6b7c3c5317545cff78a06270a3a4b33a2a8e2c08530e6d1\": container with ID starting with f2797904333c8ef9f6b7c3c5317545cff78a06270a3a4b33a2a8e2c08530e6d1 not found: ID does not exist" containerID="f2797904333c8ef9f6b7c3c5317545cff78a06270a3a4b33a2a8e2c08530e6d1"
Jul 15 16:56:18.610011 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:18.609956    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2797904333c8ef9f6b7c3c5317545cff78a06270a3a4b33a2a8e2c08530e6d1"} err="failed to get container status \"f2797904333c8ef9f6b7c3c5317545cff78a06270a3a4b33a2a8e2c08530e6d1\": rpc error: code = NotFound desc = could not find container \"f2797904333c8ef9f6b7c3c5317545cff78a06270a3a4b33a2a8e2c08530e6d1\": container with ID starting with f2797904333c8ef9f6b7c3c5317545cff78a06270a3a4b33a2a8e2c08530e6d1 not found: ID does not exist"
Jul 15 16:56:18.630679 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:18.630651    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q"]
Jul 15 16:56:18.633471 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:18.633447    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-cd9127-7c849dff6-rxq4q"]
Jul 15 16:56:18.665791 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:18.665770    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e02f4cd-627f-4778-ba5e-f5b31b933518-openshift-service-ca\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:56:18.665875 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:18.665811    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e02f4cd-627f-4778-ba5e-f5b31b933518-maas-api-tls\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:56:18.665875 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:18.665821    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twpp7\" (UniqueName: \"kubernetes.io/projected/3e02f4cd-627f-4778-ba5e-f5b31b933518-kube-api-access-twpp7\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:56:18.878663 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:18.878583    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e02f4cd-627f-4778-ba5e-f5b31b933518" path="/var/lib/kubelet/pods/3e02f4cd-627f-4778-ba5e-f5b31b933518/volumes"
Jul 15 16:56:21.380680 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:21.380643    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:56:22.785860 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:22.785827    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:28.770440 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:28.770405    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:28.838034 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:28.837995    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:33.519024 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:33.518979    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:33.738239 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:33.738201    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:34.079934 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:34.079891    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:34.419790 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:34.419683    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:34.617945 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:34.617903    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:37.574031 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.573991    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:37.676600 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.676555    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:37.687897 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.687868    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv"]
Jul 15 16:56:37.688222 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.688208    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e02f4cd-627f-4778-ba5e-f5b31b933518" containerName="maas-api"
Jul 15 16:56:37.688222 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.688223    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e02f4cd-627f-4778-ba5e-f5b31b933518" containerName="maas-api"
Jul 15 16:56:37.688351 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.688233    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecfc41f1-5028-491f-8d95-cd3a2564b841" containerName="revoke-keys"
Jul 15 16:56:37.688351 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.688240    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecfc41f1-5028-491f-8d95-cd3a2564b841" containerName="revoke-keys"
Jul 15 16:56:37.688351 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.688306    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecfc41f1-5028-491f-8d95-cd3a2564b841" containerName="revoke-keys"
Jul 15 16:56:37.688351 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.688316    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e02f4cd-627f-4778-ba5e-f5b31b933518" containerName="maas-api"
Jul 15 16:56:37.695358 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.695333    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv"
Jul 15 16:56:37.698692 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.698668    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-v8jb8\""
Jul 15 16:56:37.698932 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.698916    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 16:56:37.699552 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.699535    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 16:56:37.700295 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.700277    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-00e709-serving-cert\""
Jul 15 16:56:37.713689 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.713666    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv"]
Jul 15 16:56:37.770092 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.770053    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:37.825493 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.825408    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4031608-3cd5-4777-965c-033456437ba4-maas-api-tls\") pod \"maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv\" (UID: \"b4031608-3cd5-4777-965c-033456437ba4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv"
Jul 15 16:56:37.825493 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.825459    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j48f\" (UniqueName: \"kubernetes.io/projected/b4031608-3cd5-4777-965c-033456437ba4-kube-api-access-8j48f\") pod \"maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv\" (UID: \"b4031608-3cd5-4777-965c-033456437ba4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv"
Jul 15 16:56:37.825682 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.825531    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4031608-3cd5-4777-965c-033456437ba4-openshift-service-ca\") pod \"maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv\" (UID: \"b4031608-3cd5-4777-965c-033456437ba4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv"
Jul 15 16:56:37.925185 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.925147    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:37.926110 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.926089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j48f\" (UniqueName: \"kubernetes.io/projected/b4031608-3cd5-4777-965c-033456437ba4-kube-api-access-8j48f\") pod \"maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv\" (UID: \"b4031608-3cd5-4777-965c-033456437ba4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv"
Jul 15 16:56:37.926181 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.926140    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4031608-3cd5-4777-965c-033456437ba4-openshift-service-ca\") pod \"maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv\" (UID: \"b4031608-3cd5-4777-965c-033456437ba4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv"
Jul 15 16:56:37.926219 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.926182    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4031608-3cd5-4777-965c-033456437ba4-maas-api-tls\") pod \"maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv\" (UID: \"b4031608-3cd5-4777-965c-033456437ba4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv"
Jul 15 16:56:37.926986 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.926959    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4031608-3cd5-4777-965c-033456437ba4-openshift-service-ca\") pod \"maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv\" (UID: \"b4031608-3cd5-4777-965c-033456437ba4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv"
Jul 15 16:56:37.928567 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.928546    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4031608-3cd5-4777-965c-033456437ba4-maas-api-tls\") pod \"maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv\" (UID: \"b4031608-3cd5-4777-965c-033456437ba4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv"
Jul 15 16:56:37.935668 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:37.935648    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j48f\" (UniqueName: \"kubernetes.io/projected/b4031608-3cd5-4777-965c-033456437ba4-kube-api-access-8j48f\") pod \"maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv\" (UID: \"b4031608-3cd5-4777-965c-033456437ba4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv"
Jul 15 16:56:38.005724 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:38.005682    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv"
Jul 15 16:56:38.149419 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:38.149385    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv"]
Jul 15 16:56:38.152269 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:56:38.152240    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4031608_3cd5_4777_965c_033456437ba4.slice/crio-1fd9ad403637368b7bf70ee63e3840da2c23fbb9813862f059b536ea2f783d8e WatchSource:0}: Error finding container 1fd9ad403637368b7bf70ee63e3840da2c23fbb9813862f059b536ea2f783d8e: Status 404 returned error can't find the container with id 1fd9ad403637368b7bf70ee63e3840da2c23fbb9813862f059b536ea2f783d8e
Jul 15 16:56:38.308124 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:38.308085    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:38.407122 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:38.407018    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:38.702530 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:38.702440    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv" event={"ID":"b4031608-3cd5-4777-965c-033456437ba4","Type":"ContainerStarted","Data":"2978e1bc3b31ebe7205f38380e5b7ead8ab84c7d9285531e4679f38619c61da9"}
Jul 15 16:56:38.702530 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:38.702491    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv" event={"ID":"b4031608-3cd5-4777-965c-033456437ba4","Type":"ContainerStarted","Data":"1fd9ad403637368b7bf70ee63e3840da2c23fbb9813862f059b536ea2f783d8e"}
Jul 15 16:56:38.702958 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:38.702539    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv"
Jul 15 16:56:38.725219 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:38.725164    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv" podStartSLOduration=1.3462590190000001 podStartE2EDuration="1.725144504s" totalImagesPullingTime="378.885485ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:56:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:38.723734452 +0000 UTC m=+2400.370774989" watchObservedRunningTime="2026-07-15 16:56:38.725144504 +0000 UTC m=+2400.372185045"
Jul 15 16:56:38.907584 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:38.907556    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-295d8_46356463-bfb1-42b8-8871-71357489a95b/ovn-acl-logging/0.log"
Jul 15 16:56:38.912941 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:38.912920    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-295d8_46356463-bfb1-42b8-8871-71357489a95b/ovn-acl-logging/0.log"
Jul 15 16:56:39.759084 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:39.759047    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:39.858418 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:39.858383    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:40.555160 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:40.555126    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:41.716910 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:41.716870    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:42.117038 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:42.117003    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:42.589671 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:42.589633    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:42.895843 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:42.895746    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:56:42.921971 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:42.921927    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:43.066338 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:43.066297    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:44.713302 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:44.713267    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv"
Jul 15 16:56:46.124963 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:46.124926    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:46.207170 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:46.207132    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp"]
Jul 15 16:56:46.210691 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:46.210662    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp"
Jul 15 16:56:46.213646 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:46.213623    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-90f61e-serving-cert\""
Jul 15 16:56:46.223615 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:46.223579    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:46.224367 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:46.224340    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp"]
Jul 15 16:56:46.296484 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:46.296452    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2548b\" (UniqueName: \"kubernetes.io/projected/b5ca7e82-3826-425e-9a8d-fd6d5299e578-kube-api-access-2548b\") pod \"maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp\" (UID: \"b5ca7e82-3826-425e-9a8d-fd6d5299e578\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp"
Jul 15 16:56:46.296632 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:46.296492    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5ca7e82-3826-425e-9a8d-fd6d5299e578-maas-api-tls\") pod \"maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp\" (UID: \"b5ca7e82-3826-425e-9a8d-fd6d5299e578\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp"
Jul 15 16:56:46.296632 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:46.296523    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5ca7e82-3826-425e-9a8d-fd6d5299e578-openshift-service-ca\") pod \"maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp\" (UID: \"b5ca7e82-3826-425e-9a8d-fd6d5299e578\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp"
Jul 15 16:56:46.325003 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:46.324966    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:46.397478 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:46.397386    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2548b\" (UniqueName: \"kubernetes.io/projected/b5ca7e82-3826-425e-9a8d-fd6d5299e578-kube-api-access-2548b\") pod \"maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp\" (UID: \"b5ca7e82-3826-425e-9a8d-fd6d5299e578\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp"
Jul 15 16:56:46.397478 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:46.397429    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5ca7e82-3826-425e-9a8d-fd6d5299e578-maas-api-tls\") pod \"maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp\" (UID: \"b5ca7e82-3826-425e-9a8d-fd6d5299e578\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp"
Jul 15 16:56:46.397684 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:46.397487    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5ca7e82-3826-425e-9a8d-fd6d5299e578-openshift-service-ca\") pod \"maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp\" (UID: \"b5ca7e82-3826-425e-9a8d-fd6d5299e578\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp"
Jul 15 16:56:46.397684 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:56:46.397533    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-serving-cert: secret "maas-api-e2e-api-b-90f61e-serving-cert" not found
Jul 15 16:56:46.397684 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:56:46.397588    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5ca7e82-3826-425e-9a8d-fd6d5299e578-maas-api-tls podName:b5ca7e82-3826-425e-9a8d-fd6d5299e578 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:46.897570185 +0000 UTC m=+2408.544610699 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b5ca7e82-3826-425e-9a8d-fd6d5299e578-maas-api-tls") pod "maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp" (UID: "b5ca7e82-3826-425e-9a8d-fd6d5299e578") : secret "maas-api-e2e-api-b-90f61e-serving-cert" not found
Jul 15 16:56:46.398326 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:46.398309    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5ca7e82-3826-425e-9a8d-fd6d5299e578-openshift-service-ca\") pod \"maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp\" (UID: \"b5ca7e82-3826-425e-9a8d-fd6d5299e578\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp"
Jul 15 16:56:46.409029 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:46.409006    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2548b\" (UniqueName: \"kubernetes.io/projected/b5ca7e82-3826-425e-9a8d-fd6d5299e578-kube-api-access-2548b\") pod \"maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp\" (UID: \"b5ca7e82-3826-425e-9a8d-fd6d5299e578\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp"
Jul 15 16:56:46.819695 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:46.819659    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:46.902578 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:46.902542    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5ca7e82-3826-425e-9a8d-fd6d5299e578-maas-api-tls\") pod \"maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp\" (UID: \"b5ca7e82-3826-425e-9a8d-fd6d5299e578\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp"
Jul 15 16:56:46.902779 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:56:46.902687    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-serving-cert: secret "maas-api-e2e-api-b-90f61e-serving-cert" not found
Jul 15 16:56:46.902835 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:56:46.902779    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5ca7e82-3826-425e-9a8d-fd6d5299e578-maas-api-tls podName:b5ca7e82-3826-425e-9a8d-fd6d5299e578 nodeName:}" failed. No retries permitted until 2026-07-15 16:56:47.902742888 +0000 UTC m=+2409.549783402 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b5ca7e82-3826-425e-9a8d-fd6d5299e578-maas-api-tls") pod "maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp" (UID: "b5ca7e82-3826-425e-9a8d-fd6d5299e578") : secret "maas-api-e2e-api-b-90f61e-serving-cert" not found
Jul 15 16:56:46.918973 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:46.918933    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:47.529042 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:47.529002    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:47.911110 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:47.911077    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5ca7e82-3826-425e-9a8d-fd6d5299e578-maas-api-tls\") pod \"maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp\" (UID: \"b5ca7e82-3826-425e-9a8d-fd6d5299e578\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp"
Jul 15 16:56:47.913527 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:47.913500    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5ca7e82-3826-425e-9a8d-fd6d5299e578-maas-api-tls\") pod \"maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp\" (UID: \"b5ca7e82-3826-425e-9a8d-fd6d5299e578\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp"
Jul 15 16:56:48.024089 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:48.024042    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp"
Jul 15 16:56:48.170724 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:48.170698    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp"]
Jul 15 16:56:48.172533 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:56:48.172498    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5ca7e82_3826_425e_9a8d_fd6d5299e578.slice/crio-fc019f3165f340420087964fea7efc787102e75fc858cae6f725ac9e385663b9 WatchSource:0}: Error finding container fc019f3165f340420087964fea7efc787102e75fc858cae6f725ac9e385663b9: Status 404 returned error can't find the container with id fc019f3165f340420087964fea7efc787102e75fc858cae6f725ac9e385663b9
Jul 15 16:56:48.763845 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:48.763804    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp" event={"ID":"b5ca7e82-3826-425e-9a8d-fd6d5299e578","Type":"ContainerStarted","Data":"37ccbb5e32d35e8bcb45b88f6e7dbc5e5878e0ab5782411199414ebe4f4b979e"}
Jul 15 16:56:48.764674 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:48.763900    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp" event={"ID":"b5ca7e82-3826-425e-9a8d-fd6d5299e578","Type":"ContainerStarted","Data":"fc019f3165f340420087964fea7efc787102e75fc858cae6f725ac9e385663b9"}
Jul 15 16:56:48.764674 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:48.763930    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp"
Jul 15 16:56:48.784236 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:48.784188    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp" podStartSLOduration=2.408824843 podStartE2EDuration="2.784174036s" totalImagesPullingTime="375.349193ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:56:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:48.782187212 +0000 UTC m=+2410.429227748" watchObservedRunningTime="2026-07-15 16:56:48.784174036 +0000 UTC m=+2410.431214571"
Jul 15 16:56:50.596510 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:50.596478    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:53.708012 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:53.707974    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:56:54.775556 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:54.775525    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp"
Jul 15 16:56:58.968015 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:58.967979    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-00e709-gtsgc"]
Jul 15 16:56:58.971678 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:58.971661    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-00e709-gtsgc"
Jul 15 16:56:58.975100 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:58.975074    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lsp65\""
Jul 15 16:56:58.980463 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:58.980438    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-00e709-gtsgc"]
Jul 15 16:56:59.109956 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:59.109917    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/92e59ceb-5db3-46c3-a465-316af0c3b0ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-00e709-gtsgc\" (UID: \"92e59ceb-5db3-46c3-a465-316af0c3b0ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-00e709-gtsgc"
Jul 15 16:56:59.211103 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:59.211063    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/92e59ceb-5db3-46c3-a465-316af0c3b0ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-00e709-gtsgc\" (UID: \"92e59ceb-5db3-46c3-a465-316af0c3b0ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-00e709-gtsgc"
Jul 15 16:56:59.211698 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:59.211676    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/92e59ceb-5db3-46c3-a465-316af0c3b0ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-00e709-gtsgc\" (UID: \"92e59ceb-5db3-46c3-a465-316af0c3b0ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-00e709-gtsgc"
Jul 15 16:56:59.283350 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:59.283265    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-00e709-gtsgc"
Jul 15 16:56:59.419224 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:59.419200    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-00e709-gtsgc"]
Jul 15 16:56:59.421526 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:56:59.421498    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92e59ceb_5db3_46c3_a465_316af0c3b0ba.slice/crio-f0cb3befe4f286be130011e884a3d131b72f8eda1cd02f599f1193ea4eb7a15d WatchSource:0}: Error finding container f0cb3befe4f286be130011e884a3d131b72f8eda1cd02f599f1193ea4eb7a15d: Status 404 returned error can't find the container with id f0cb3befe4f286be130011e884a3d131b72f8eda1cd02f599f1193ea4eb7a15d
Jul 15 16:56:59.540357 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:59.540279    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:59.653606 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:59.653576    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:56:59.819672 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:59.819640    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-00e709-gtsgc" event={"ID":"92e59ceb-5db3-46c3-a465-316af0c3b0ba","Type":"ContainerStarted","Data":"d019efd8729ff5b2ee80d2a9db3a3579beb9d2047b518d7c92cf1458eceec7a7"}
Jul 15 16:56:59.819884 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:59.819696    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-00e709-gtsgc" event={"ID":"92e59ceb-5db3-46c3-a465-316af0c3b0ba","Type":"ContainerStarted","Data":"f0cb3befe4f286be130011e884a3d131b72f8eda1cd02f599f1193ea4eb7a15d"}
Jul 15 16:56:59.841803 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:56:59.841731    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-00e709-gtsgc" podStartSLOduration=1.8417120809999998 podStartE2EDuration="1.841712081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:56:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:56:59.840702296 +0000 UTC m=+2421.487742833" watchObservedRunningTime="2026-07-15 16:56:59.841712081 +0000 UTC m=+2421.488752617"
Jul 15 16:57:00.832268 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:00.832226    2580 generic.go:356] "Generic (PLEG): container finished" podID="92e59ceb-5db3-46c3-a465-316af0c3b0ba" containerID="d019efd8729ff5b2ee80d2a9db3a3579beb9d2047b518d7c92cf1458eceec7a7" exitCode=0
Jul 15 16:57:00.832676 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:00.832318    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-00e709-gtsgc" event={"ID":"92e59ceb-5db3-46c3-a465-316af0c3b0ba","Type":"ContainerDied","Data":"d019efd8729ff5b2ee80d2a9db3a3579beb9d2047b518d7c92cf1458eceec7a7"}
Jul 15 16:57:01.988530 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:01.988504    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-00e709-gtsgc"
Jul 15 16:57:02.137512 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:02.137420    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/92e59ceb-5db3-46c3-a465-316af0c3b0ba-maas-api-service-ca\") pod \"92e59ceb-5db3-46c3-a465-316af0c3b0ba\" (UID: \"92e59ceb-5db3-46c3-a465-316af0c3b0ba\") "
Jul 15 16:57:02.137819 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:02.137750    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92e59ceb-5db3-46c3-a465-316af0c3b0ba-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "92e59ceb-5db3-46c3-a465-316af0c3b0ba" (UID: "92e59ceb-5db3-46c3-a465-316af0c3b0ba"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:57:02.238774 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:02.238735    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/92e59ceb-5db3-46c3-a465-316af0c3b0ba-maas-api-service-ca\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:57:02.844094 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:02.844064    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-00e709-gtsgc"
Jul 15 16:57:02.844275 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:02.844103    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-00e709-gtsgc" event={"ID":"92e59ceb-5db3-46c3-a465-316af0c3b0ba","Type":"ContainerDied","Data":"f0cb3befe4f286be130011e884a3d131b72f8eda1cd02f599f1193ea4eb7a15d"}
Jul 15 16:57:02.844275 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:02.844149    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f0cb3befe4f286be130011e884a3d131b72f8eda1cd02f599f1193ea4eb7a15d"
Jul 15 16:57:07.687503 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:07.687467    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:07.791048 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:07.791011    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:09.578145 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:09.578098    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:09.823630 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:09.823593    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv"]
Jul 15 16:57:09.824157 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:09.824120    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv" podUID="b4031608-3cd5-4777-965c-033456437ba4" containerName="maas-api" containerID="cri-o://2978e1bc3b31ebe7205f38380e5b7ead8ab84c7d9285531e4679f38619c61da9" gracePeriod=30
Jul 15 16:57:10.076128 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:10.076102    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv"
Jul 15 16:57:10.104607 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:10.104533    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4031608-3cd5-4777-965c-033456437ba4-openshift-service-ca\") pod \"b4031608-3cd5-4777-965c-033456437ba4\" (UID: \"b4031608-3cd5-4777-965c-033456437ba4\") "
Jul 15 16:57:10.104607 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:10.104575    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8j48f\" (UniqueName: \"kubernetes.io/projected/b4031608-3cd5-4777-965c-033456437ba4-kube-api-access-8j48f\") pod \"b4031608-3cd5-4777-965c-033456437ba4\" (UID: \"b4031608-3cd5-4777-965c-033456437ba4\") "
Jul 15 16:57:10.104784 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:10.104608    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4031608-3cd5-4777-965c-033456437ba4-maas-api-tls\") pod \"b4031608-3cd5-4777-965c-033456437ba4\" (UID: \"b4031608-3cd5-4777-965c-033456437ba4\") "
Jul 15 16:57:10.104975 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:10.104949    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4031608-3cd5-4777-965c-033456437ba4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b4031608-3cd5-4777-965c-033456437ba4" (UID: "b4031608-3cd5-4777-965c-033456437ba4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:57:10.106702 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:10.106675    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4031608-3cd5-4777-965c-033456437ba4-kube-api-access-8j48f" (OuterVolumeSpecName: "kube-api-access-8j48f") pod "b4031608-3cd5-4777-965c-033456437ba4" (UID: "b4031608-3cd5-4777-965c-033456437ba4"). InnerVolumeSpecName "kube-api-access-8j48f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:57:10.106924 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:10.106716    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4031608-3cd5-4777-965c-033456437ba4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b4031608-3cd5-4777-965c-033456437ba4" (UID: "b4031608-3cd5-4777-965c-033456437ba4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:57:10.205846 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:10.205808    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4031608-3cd5-4777-965c-033456437ba4-openshift-service-ca\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:57:10.205846 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:10.205841    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8j48f\" (UniqueName: \"kubernetes.io/projected/b4031608-3cd5-4777-965c-033456437ba4-kube-api-access-8j48f\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:57:10.205846 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:10.205851    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4031608-3cd5-4777-965c-033456437ba4-maas-api-tls\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:57:10.435966 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:10.435856    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:10.536952 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:10.536920    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:10.885271 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:10.885237    2580 generic.go:356] "Generic (PLEG): container finished" podID="b4031608-3cd5-4777-965c-033456437ba4" containerID="2978e1bc3b31ebe7205f38380e5b7ead8ab84c7d9285531e4679f38619c61da9" exitCode=0
Jul 15 16:57:10.885271 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:10.885274    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv"
Jul 15 16:57:10.885722 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:10.885307    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv" event={"ID":"b4031608-3cd5-4777-965c-033456437ba4","Type":"ContainerDied","Data":"2978e1bc3b31ebe7205f38380e5b7ead8ab84c7d9285531e4679f38619c61da9"}
Jul 15 16:57:10.885722 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:10.885359    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv" event={"ID":"b4031608-3cd5-4777-965c-033456437ba4","Type":"ContainerDied","Data":"1fd9ad403637368b7bf70ee63e3840da2c23fbb9813862f059b536ea2f783d8e"}
Jul 15 16:57:10.885722 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:10.885391    2580 scope.go:117] "RemoveContainer" containerID="2978e1bc3b31ebe7205f38380e5b7ead8ab84c7d9285531e4679f38619c61da9"
Jul 15 16:57:10.894976 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:10.894957    2580 scope.go:117] "RemoveContainer" containerID="2978e1bc3b31ebe7205f38380e5b7ead8ab84c7d9285531e4679f38619c61da9"
Jul 15 16:57:10.895249 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:57:10.895228    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2978e1bc3b31ebe7205f38380e5b7ead8ab84c7d9285531e4679f38619c61da9\": container with ID starting with 2978e1bc3b31ebe7205f38380e5b7ead8ab84c7d9285531e4679f38619c61da9 not found: ID does not exist" containerID="2978e1bc3b31ebe7205f38380e5b7ead8ab84c7d9285531e4679f38619c61da9"
Jul 15 16:57:10.895307 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:10.895260    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2978e1bc3b31ebe7205f38380e5b7ead8ab84c7d9285531e4679f38619c61da9"} err="failed to get container status \"2978e1bc3b31ebe7205f38380e5b7ead8ab84c7d9285531e4679f38619c61da9\": rpc error: code = NotFound desc = could not find container \"2978e1bc3b31ebe7205f38380e5b7ead8ab84c7d9285531e4679f38619c61da9\": container with ID starting with 2978e1bc3b31ebe7205f38380e5b7ead8ab84c7d9285531e4679f38619c61da9 not found: ID does not exist"
Jul 15 16:57:10.906931 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:10.906900    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv"]
Jul 15 16:57:10.910931 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:10.910908    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-00e709-8666bd5b67-q8jfv"]
Jul 15 16:57:12.878806 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:12.878773    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4031608-3cd5-4777-965c-033456437ba4" path="/var/lib/kubelet/pods/b4031608-3cd5-4777-965c-033456437ba4/volumes"
Jul 15 16:57:13.289694 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:13.284504    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:57:14.084938 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:14.084900    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:19.794503 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:19.794456    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:19.894456 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:19.894421    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:24.813589 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:24.813551    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:25.071649 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:25.071532    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:25.194399 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:25.194369    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-90f61e-2bx6j"]
Jul 15 16:57:25.194733 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:25.194721    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4031608-3cd5-4777-965c-033456437ba4" containerName="maas-api"
Jul 15 16:57:25.194812 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:25.194735    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4031608-3cd5-4777-965c-033456437ba4" containerName="maas-api"
Jul 15 16:57:25.194812 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:25.194747    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92e59ceb-5db3-46c3-a465-316af0c3b0ba" containerName="revoke-keys"
Jul 15 16:57:25.194812 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:25.194766    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="92e59ceb-5db3-46c3-a465-316af0c3b0ba" containerName="revoke-keys"
Jul 15 16:57:25.194911 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:25.194822    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="92e59ceb-5db3-46c3-a465-316af0c3b0ba" containerName="revoke-keys"
Jul 15 16:57:25.194911 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:25.194832    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4031608-3cd5-4777-965c-033456437ba4" containerName="maas-api"
Jul 15 16:57:25.197808 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:25.197791    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-90f61e-2bx6j"
Jul 15 16:57:25.200777 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:25.200747    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lsp65\""
Jul 15 16:57:25.206340 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:25.206311    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-90f61e-2bx6j"]
Jul 15 16:57:25.229394 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:25.229367    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab2c263f-62a3-40c4-b394-6d68d26f6453-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-90f61e-2bx6j\" (UID: \"ab2c263f-62a3-40c4-b394-6d68d26f6453\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-90f61e-2bx6j"
Jul 15 16:57:25.330748 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:25.330661    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab2c263f-62a3-40c4-b394-6d68d26f6453-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-90f61e-2bx6j\" (UID: \"ab2c263f-62a3-40c4-b394-6d68d26f6453\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-90f61e-2bx6j"
Jul 15 16:57:25.331554 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:25.331532    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab2c263f-62a3-40c4-b394-6d68d26f6453-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-90f61e-2bx6j\" (UID: \"ab2c263f-62a3-40c4-b394-6d68d26f6453\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-90f61e-2bx6j"
Jul 15 16:57:25.508583 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:25.508542    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-90f61e-2bx6j"
Jul 15 16:57:25.637131 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:25.637096    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-90f61e-2bx6j"]
Jul 15 16:57:25.638266 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:57:25.638238    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab2c263f_62a3_40c4_b394_6d68d26f6453.slice/crio-f94096a54d73d12998ccfb6005c176da353ba3d23277bc991c3be465e83bc6e6 WatchSource:0}: Error finding container f94096a54d73d12998ccfb6005c176da353ba3d23277bc991c3be465e83bc6e6: Status 404 returned error can't find the container with id f94096a54d73d12998ccfb6005c176da353ba3d23277bc991c3be465e83bc6e6
Jul 15 16:57:25.782099 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:25.782064    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:25.882339 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:25.882299    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:25.962080 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:25.962042    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-90f61e-2bx6j" event={"ID":"ab2c263f-62a3-40c4-b394-6d68d26f6453","Type":"ContainerStarted","Data":"c80000a2aa70a1d44f843895bf1fe8c5da913811bac671f45cbec58d88e8ef6d"}
Jul 15 16:57:25.962244 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:25.962124    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-90f61e-2bx6j" event={"ID":"ab2c263f-62a3-40c4-b394-6d68d26f6453","Type":"ContainerStarted","Data":"f94096a54d73d12998ccfb6005c176da353ba3d23277bc991c3be465e83bc6e6"}
Jul 15 16:57:25.978625 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:25.978575    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-90f61e-2bx6j" podStartSLOduration=0.978558584 podStartE2EDuration="978.558584ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:57:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:57:25.977432769 +0000 UTC m=+2447.624473306" watchObservedRunningTime="2026-07-15 16:57:25.978558584 +0000 UTC m=+2447.625599145"
Jul 15 16:57:26.968421 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:26.968385    2580 generic.go:356] "Generic (PLEG): container finished" podID="ab2c263f-62a3-40c4-b394-6d68d26f6453" containerID="c80000a2aa70a1d44f843895bf1fe8c5da913811bac671f45cbec58d88e8ef6d" exitCode=0
Jul 15 16:57:26.968839 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:26.968456    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-90f61e-2bx6j" event={"ID":"ab2c263f-62a3-40c4-b394-6d68d26f6453","Type":"ContainerDied","Data":"c80000a2aa70a1d44f843895bf1fe8c5da913811bac671f45cbec58d88e8ef6d"}
Jul 15 16:57:28.107451 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:28.107428    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-90f61e-2bx6j"
Jul 15 16:57:28.154941 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:28.154907    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab2c263f-62a3-40c4-b394-6d68d26f6453-maas-api-service-ca\") pod \"ab2c263f-62a3-40c4-b394-6d68d26f6453\" (UID: \"ab2c263f-62a3-40c4-b394-6d68d26f6453\") "
Jul 15 16:57:28.155260 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:28.155236    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab2c263f-62a3-40c4-b394-6d68d26f6453-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ab2c263f-62a3-40c4-b394-6d68d26f6453" (UID: "ab2c263f-62a3-40c4-b394-6d68d26f6453"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:57:28.255910 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:28.255821    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab2c263f-62a3-40c4-b394-6d68d26f6453-maas-api-service-ca\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:57:28.978830 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:28.978790    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-90f61e-2bx6j" event={"ID":"ab2c263f-62a3-40c4-b394-6d68d26f6453","Type":"ContainerDied","Data":"f94096a54d73d12998ccfb6005c176da353ba3d23277bc991c3be465e83bc6e6"}
Jul 15 16:57:28.978830 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:28.978816    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-90f61e-2bx6j"
Jul 15 16:57:28.979044 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:28.978856    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f94096a54d73d12998ccfb6005c176da353ba3d23277bc991c3be465e83bc6e6"
Jul 15 16:57:35.656357 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:35.656322    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp"]
Jul 15 16:57:35.656806 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:35.656574    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp" podUID="b5ca7e82-3826-425e-9a8d-fd6d5299e578" containerName="maas-api" containerID="cri-o://37ccbb5e32d35e8bcb45b88f6e7dbc5e5878e0ab5782411199414ebe4f4b979e" gracePeriod=30
Jul 15 16:57:35.696396 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:35.696356    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:35.915820 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:35.915746    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp"
Jul 15 16:57:36.015476 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:36.015440    2580 generic.go:356] "Generic (PLEG): container finished" podID="b5ca7e82-3826-425e-9a8d-fd6d5299e578" containerID="37ccbb5e32d35e8bcb45b88f6e7dbc5e5878e0ab5782411199414ebe4f4b979e" exitCode=0
Jul 15 16:57:36.015655 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:36.015487    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp"
Jul 15 16:57:36.015655 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:36.015528    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp" event={"ID":"b5ca7e82-3826-425e-9a8d-fd6d5299e578","Type":"ContainerDied","Data":"37ccbb5e32d35e8bcb45b88f6e7dbc5e5878e0ab5782411199414ebe4f4b979e"}
Jul 15 16:57:36.015655 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:36.015579    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp" event={"ID":"b5ca7e82-3826-425e-9a8d-fd6d5299e578","Type":"ContainerDied","Data":"fc019f3165f340420087964fea7efc787102e75fc858cae6f725ac9e385663b9"}
Jul 15 16:57:36.015655 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:36.015608    2580 scope.go:117] "RemoveContainer" containerID="37ccbb5e32d35e8bcb45b88f6e7dbc5e5878e0ab5782411199414ebe4f4b979e"
Jul 15 16:57:36.021828 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:36.021808    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5ca7e82-3826-425e-9a8d-fd6d5299e578-maas-api-tls\") pod \"b5ca7e82-3826-425e-9a8d-fd6d5299e578\" (UID: \"b5ca7e82-3826-425e-9a8d-fd6d5299e578\") "
Jul 15 16:57:36.021949 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:36.021845    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5ca7e82-3826-425e-9a8d-fd6d5299e578-openshift-service-ca\") pod \"b5ca7e82-3826-425e-9a8d-fd6d5299e578\" (UID: \"b5ca7e82-3826-425e-9a8d-fd6d5299e578\") "
Jul 15 16:57:36.021949 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:36.021905    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2548b\" (UniqueName: \"kubernetes.io/projected/b5ca7e82-3826-425e-9a8d-fd6d5299e578-kube-api-access-2548b\") pod \"b5ca7e82-3826-425e-9a8d-fd6d5299e578\" (UID: \"b5ca7e82-3826-425e-9a8d-fd6d5299e578\") "
Jul 15 16:57:36.022252 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:36.022226    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5ca7e82-3826-425e-9a8d-fd6d5299e578-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b5ca7e82-3826-425e-9a8d-fd6d5299e578" (UID: "b5ca7e82-3826-425e-9a8d-fd6d5299e578"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 16:57:36.024049 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:36.024028    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5ca7e82-3826-425e-9a8d-fd6d5299e578-kube-api-access-2548b" (OuterVolumeSpecName: "kube-api-access-2548b") pod "b5ca7e82-3826-425e-9a8d-fd6d5299e578" (UID: "b5ca7e82-3826-425e-9a8d-fd6d5299e578"). InnerVolumeSpecName "kube-api-access-2548b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 16:57:36.024205 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:36.024189    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5ca7e82-3826-425e-9a8d-fd6d5299e578-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b5ca7e82-3826-425e-9a8d-fd6d5299e578" (UID: "b5ca7e82-3826-425e-9a8d-fd6d5299e578"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 16:57:36.024996 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:36.024980    2580 scope.go:117] "RemoveContainer" containerID="37ccbb5e32d35e8bcb45b88f6e7dbc5e5878e0ab5782411199414ebe4f4b979e"
Jul 15 16:57:36.025294 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:57:36.025266    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37ccbb5e32d35e8bcb45b88f6e7dbc5e5878e0ab5782411199414ebe4f4b979e\": container with ID starting with 37ccbb5e32d35e8bcb45b88f6e7dbc5e5878e0ab5782411199414ebe4f4b979e not found: ID does not exist" containerID="37ccbb5e32d35e8bcb45b88f6e7dbc5e5878e0ab5782411199414ebe4f4b979e"
Jul 15 16:57:36.025362 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:36.025303    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37ccbb5e32d35e8bcb45b88f6e7dbc5e5878e0ab5782411199414ebe4f4b979e"} err="failed to get container status \"37ccbb5e32d35e8bcb45b88f6e7dbc5e5878e0ab5782411199414ebe4f4b979e\": rpc error: code = NotFound desc = could not find container \"37ccbb5e32d35e8bcb45b88f6e7dbc5e5878e0ab5782411199414ebe4f4b979e\": container with ID starting with 37ccbb5e32d35e8bcb45b88f6e7dbc5e5878e0ab5782411199414ebe4f4b979e not found: ID does not exist"
Jul 15 16:57:36.122893 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:36.122856    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5ca7e82-3826-425e-9a8d-fd6d5299e578-maas-api-tls\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:57:36.122893 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:36.122892    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5ca7e82-3826-425e-9a8d-fd6d5299e578-openshift-service-ca\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:57:36.123137 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:36.122907    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2548b\" (UniqueName: \"kubernetes.io/projected/b5ca7e82-3826-425e-9a8d-fd6d5299e578-kube-api-access-2548b\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 16:57:36.339545 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:36.339513    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp"]
Jul 15 16:57:36.344043 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:36.344017    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-90f61e-7d8c9c6988-vsvcp"]
Jul 15 16:57:36.878700 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:36.878662    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5ca7e82-3826-425e-9a8d-fd6d5299e578" path="/var/lib/kubelet/pods/b5ca7e82-3826-425e-9a8d-fd6d5299e578/volumes"
Jul 15 16:57:38.977457 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:38.977420    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:57:39.779799 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:39.779746    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:39.880356 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:39.880320    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:40.299522 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:40.299488    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:46.678387 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:46.678343    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:46.762536 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:46.762495    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:51.038874 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:51.038820    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:51.275769 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:51.275726    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:51.644098 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:51.644058    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:51.994138 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:51.994054    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:52.273495 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:52.273406    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:55.048778 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.048720    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:55.087296 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.087267    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92"]
Jul 15 16:57:55.087738 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.087699    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab2c263f-62a3-40c4-b394-6d68d26f6453" containerName="revoke-keys"
Jul 15 16:57:55.087738 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.087723    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab2c263f-62a3-40c4-b394-6d68d26f6453" containerName="revoke-keys"
Jul 15 16:57:55.087998 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.087766    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5ca7e82-3826-425e-9a8d-fd6d5299e578" containerName="maas-api"
Jul 15 16:57:55.087998 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.087775    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5ca7e82-3826-425e-9a8d-fd6d5299e578" containerName="maas-api"
Jul 15 16:57:55.087998 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.087864    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5ca7e82-3826-425e-9a8d-fd6d5299e578" containerName="maas-api"
Jul 15 16:57:55.087998 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.087878    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab2c263f-62a3-40c4-b394-6d68d26f6453" containerName="revoke-keys"
Jul 15 16:57:55.091207 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.091186    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92"
Jul 15 16:57:55.093907 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.093885    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-v8jb8\""
Jul 15 16:57:55.094092 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.094078    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 16:57:55.094173 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.094137    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-8269b5-serving-cert\""
Jul 15 16:57:55.094809 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.094791    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 16:57:55.104684 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.104653    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92"]
Jul 15 16:57:55.140443 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.140401    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:55.187719 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.187684    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a363a7f-1088-4ad2-8a32-3c53a07164aa-maas-api-tls\") pod \"maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92\" (UID: \"7a363a7f-1088-4ad2-8a32-3c53a07164aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92"
Jul 15 16:57:55.187898 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.187735    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a363a7f-1088-4ad2-8a32-3c53a07164aa-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92\" (UID: \"7a363a7f-1088-4ad2-8a32-3c53a07164aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92"
Jul 15 16:57:55.187898 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.187870    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6v6l\" (UniqueName: \"kubernetes.io/projected/7a363a7f-1088-4ad2-8a32-3c53a07164aa-kube-api-access-d6v6l\") pod \"maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92\" (UID: \"7a363a7f-1088-4ad2-8a32-3c53a07164aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92"
Jul 15 16:57:55.289111 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.289071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6v6l\" (UniqueName: \"kubernetes.io/projected/7a363a7f-1088-4ad2-8a32-3c53a07164aa-kube-api-access-d6v6l\") pod \"maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92\" (UID: \"7a363a7f-1088-4ad2-8a32-3c53a07164aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92"
Jul 15 16:57:55.289111 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.289118    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a363a7f-1088-4ad2-8a32-3c53a07164aa-maas-api-tls\") pod \"maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92\" (UID: \"7a363a7f-1088-4ad2-8a32-3c53a07164aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92"
Jul 15 16:57:55.289399 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.289147    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a363a7f-1088-4ad2-8a32-3c53a07164aa-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92\" (UID: \"7a363a7f-1088-4ad2-8a32-3c53a07164aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92"
Jul 15 16:57:55.289399 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:57:55.289239    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-serving-cert: secret "maas-api-e2e-shared-a-8269b5-serving-cert" not found
Jul 15 16:57:55.289399 ip-10-0-134-132 kubenswrapper[2580]: E0715 16:57:55.289324    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a363a7f-1088-4ad2-8a32-3c53a07164aa-maas-api-tls podName:7a363a7f-1088-4ad2-8a32-3c53a07164aa nodeName:}" failed. No retries permitted until 2026-07-15 16:57:55.789306228 +0000 UTC m=+2477.436346743 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7a363a7f-1088-4ad2-8a32-3c53a07164aa-maas-api-tls") pod "maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92" (UID: "7a363a7f-1088-4ad2-8a32-3c53a07164aa") : secret "maas-api-e2e-shared-a-8269b5-serving-cert" not found
Jul 15 16:57:55.289727 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.289710    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a363a7f-1088-4ad2-8a32-3c53a07164aa-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92\" (UID: \"7a363a7f-1088-4ad2-8a32-3c53a07164aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92"
Jul 15 16:57:55.298277 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.298248    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6v6l\" (UniqueName: \"kubernetes.io/projected/7a363a7f-1088-4ad2-8a32-3c53a07164aa-kube-api-access-d6v6l\") pod \"maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92\" (UID: \"7a363a7f-1088-4ad2-8a32-3c53a07164aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92"
Jul 15 16:57:55.678709 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.678667    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:55.792960 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.792913    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:55.793149 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.793029    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a363a7f-1088-4ad2-8a32-3c53a07164aa-maas-api-tls\") pod \"maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92\" (UID: \"7a363a7f-1088-4ad2-8a32-3c53a07164aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92"
Jul 15 16:57:55.795483 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.795460    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a363a7f-1088-4ad2-8a32-3c53a07164aa-maas-api-tls\") pod \"maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92\" (UID: \"7a363a7f-1088-4ad2-8a32-3c53a07164aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92"
Jul 15 16:57:55.928508 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:55.928469    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:56.001426 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:56.001341    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92"
Jul 15 16:57:56.138536 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:56.138505    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92"]
Jul 15 16:57:56.140435 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:57:56.140404    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a363a7f_1088_4ad2_8a32_3c53a07164aa.slice/crio-5bb2f24abb03f312b6bb311ff89d4c3516e54b99432be6969318ff800994179a WatchSource:0}: Error finding container 5bb2f24abb03f312b6bb311ff89d4c3516e54b99432be6969318ff800994179a: Status 404 returned error can't find the container with id 5bb2f24abb03f312b6bb311ff89d4c3516e54b99432be6969318ff800994179a
Jul 15 16:57:57.132704 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:57.132660    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92" event={"ID":"7a363a7f-1088-4ad2-8a32-3c53a07164aa","Type":"ContainerStarted","Data":"1c57cc65cdac604a9c0b61f1c610c09f8b74bc07cc667ad2e2b82043f3808198"}
Jul 15 16:57:57.132938 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:57.132744    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92" event={"ID":"7a363a7f-1088-4ad2-8a32-3c53a07164aa","Type":"ContainerStarted","Data":"5bb2f24abb03f312b6bb311ff89d4c3516e54b99432be6969318ff800994179a"}
Jul 15 16:57:57.132938 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:57.132838    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92"
Jul 15 16:57:57.151305 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:57.151217    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92" podStartSLOduration=1.810760053 podStartE2EDuration="2.151195882s" totalImagesPullingTime="340.435829ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:57:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:57:57.151076519 +0000 UTC m=+2478.798117056" watchObservedRunningTime="2026-07-15 16:57:57.151195882 +0000 UTC m=+2478.798236419"
Jul 15 16:57:59.491891 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:59.491848    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:57:59.543345 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:59.543303    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:57:59.949264 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:57:59.949224    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:58:00.296248 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:00.296161    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:58:00.419152 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:00.419118    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:58:03.143200 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:03.143170    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92"
Jul 15 16:58:04.166192 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:04.166151    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:58:04.299884 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:04.299849    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m"]
Jul 15 16:58:04.303718 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:04.303697    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m"
Jul 15 16:58:04.306453 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:04.306430    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-021cd5-serving-cert\""
Jul 15 16:58:04.319029 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:04.318998    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:58:04.320054 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:04.320026    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m"]
Jul 15 16:58:04.466964 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:04.466869    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b44499b-577e-4e2e-8f40-ebcf3ef5098a-maas-api-tls\") pod \"maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m\" (UID: \"1b44499b-577e-4e2e-8f40-ebcf3ef5098a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m"
Jul 15 16:58:04.466964 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:04.466887    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:58:04.466964 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:04.466951    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2w9v\" (UniqueName: \"kubernetes.io/projected/1b44499b-577e-4e2e-8f40-ebcf3ef5098a-kube-api-access-n2w9v\") pod \"maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m\" (UID: \"1b44499b-577e-4e2e-8f40-ebcf3ef5098a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m"
Jul 15 16:58:04.467186 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:04.466998    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b44499b-577e-4e2e-8f40-ebcf3ef5098a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m\" (UID: \"1b44499b-577e-4e2e-8f40-ebcf3ef5098a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m"
Jul 15 16:58:04.568189 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:04.568156    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b44499b-577e-4e2e-8f40-ebcf3ef5098a-maas-api-tls\") pod \"maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m\" (UID: \"1b44499b-577e-4e2e-8f40-ebcf3ef5098a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m"
Jul 15 16:58:04.568350 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:04.568215    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2w9v\" (UniqueName: \"kubernetes.io/projected/1b44499b-577e-4e2e-8f40-ebcf3ef5098a-kube-api-access-n2w9v\") pod \"maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m\" (UID: \"1b44499b-577e-4e2e-8f40-ebcf3ef5098a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m"
Jul 15 16:58:04.568350 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:04.568245    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b44499b-577e-4e2e-8f40-ebcf3ef5098a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m\" (UID: \"1b44499b-577e-4e2e-8f40-ebcf3ef5098a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m"
Jul 15 16:58:04.568846 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:04.568828    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b44499b-577e-4e2e-8f40-ebcf3ef5098a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m\" (UID: \"1b44499b-577e-4e2e-8f40-ebcf3ef5098a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m"
Jul 15 16:58:04.570610 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:04.570589    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b44499b-577e-4e2e-8f40-ebcf3ef5098a-maas-api-tls\") pod \"maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m\" (UID: \"1b44499b-577e-4e2e-8f40-ebcf3ef5098a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m"
Jul 15 16:58:04.579895 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:04.579871    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2w9v\" (UniqueName: \"kubernetes.io/projected/1b44499b-577e-4e2e-8f40-ebcf3ef5098a-kube-api-access-n2w9v\") pod \"maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m\" (UID: \"1b44499b-577e-4e2e-8f40-ebcf3ef5098a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m"
Jul 15 16:58:04.613986 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:04.613963    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m"
Jul 15 16:58:04.685179 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:04.685131    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:58:04.771536 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:04.771511    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m"]
Jul 15 16:58:04.773006 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:58:04.772976    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b44499b_577e_4e2e_8f40_ebcf3ef5098a.slice/crio-ee05ee1cde9eb13d0047ec3e248c03c18d339d8cebb96f496effeaa7b45d1872 WatchSource:0}: Error finding container ee05ee1cde9eb13d0047ec3e248c03c18d339d8cebb96f496effeaa7b45d1872: Status 404 returned error can't find the container with id ee05ee1cde9eb13d0047ec3e248c03c18d339d8cebb96f496effeaa7b45d1872
Jul 15 16:58:05.173159 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:05.173127    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m" event={"ID":"1b44499b-577e-4e2e-8f40-ebcf3ef5098a","Type":"ContainerStarted","Data":"40403f8f9fcfed9919e4bad2a5fce4efecefe89cec1e6b6aacd747c5e8905200"}
Jul 15 16:58:05.173529 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:05.173185    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m" event={"ID":"1b44499b-577e-4e2e-8f40-ebcf3ef5098a","Type":"ContainerStarted","Data":"ee05ee1cde9eb13d0047ec3e248c03c18d339d8cebb96f496effeaa7b45d1872"}
Jul 15 16:58:05.173529 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:05.173213    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m"
Jul 15 16:58:05.221155 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:05.221108    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m" podStartSLOduration=0.934105914 podStartE2EDuration="1.22109206s" totalImagesPullingTime="286.986146ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:58:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:58:05.220974342 +0000 UTC m=+2486.868014885" watchObservedRunningTime="2026-07-15 16:58:05.22109206 +0000 UTC m=+2486.868132596"
Jul 15 16:58:05.384462 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:05.384375    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:58:05.524086 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:05.524042    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:58:09.002581 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:09.002543    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:58:11.185047 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:11.185014    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m"
Jul 15 16:58:12.099069 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:12.099029    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:58:16.763330 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:16.763292    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:58:16.860769 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:16.860714    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:58:17.116177 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:17.116141    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:58:22.625482 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.625442    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"]
Jul 15 16:58:22.629097 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.629063    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 16:58:22.632798 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.632743    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8269b5\"/\"kube-root-ca.crt\""
Jul 15 16:58:22.632956 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.632775    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8269b5\"/\"openshift-service-ca.crt\""
Jul 15 16:58:22.632956 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.632844    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8269b5\"/\"default-dockercfg-f4lbt\""
Jul 15 16:58:22.632956 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.632844    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8269b5\"/\"auth-test-model-8269b5-kserve-self-signed-certs\""
Jul 15 16:58:22.639665 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.639641    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"]
Jul 15 16:58:22.663303 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.663270    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:58:22.735736 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.735697    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkt9l\" (UniqueName: \"kubernetes.io/projected/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-kube-api-access-pkt9l\") pod \"auth-test-model-8269b5-kserve-95f6784c-9ptdj\" (UID: \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\") " pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 16:58:22.735991 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.735750    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-tmp-dir\") pod \"auth-test-model-8269b5-kserve-95f6784c-9ptdj\" (UID: \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\") " pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 16:58:22.735991 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.735806    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-model-cache\") pod \"auth-test-model-8269b5-kserve-95f6784c-9ptdj\" (UID: \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\") " pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 16:58:22.735991 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.735897    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-home\") pod \"auth-test-model-8269b5-kserve-95f6784c-9ptdj\" (UID: \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\") " pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 16:58:22.735991 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.735947    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-tls-certs\") pod \"auth-test-model-8269b5-kserve-95f6784c-9ptdj\" (UID: \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\") " pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 16:58:22.736223 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.736037    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-dshm\") pod \"auth-test-model-8269b5-kserve-95f6784c-9ptdj\" (UID: \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\") " pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 16:58:22.837238 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.837202    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-tmp-dir\") pod \"auth-test-model-8269b5-kserve-95f6784c-9ptdj\" (UID: \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\") " pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 16:58:22.837238 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.837243    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-model-cache\") pod \"auth-test-model-8269b5-kserve-95f6784c-9ptdj\" (UID: \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\") " pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 16:58:22.837533 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.837293    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-home\") pod \"auth-test-model-8269b5-kserve-95f6784c-9ptdj\" (UID: \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\") " pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 16:58:22.837533 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.837311    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-tls-certs\") pod \"auth-test-model-8269b5-kserve-95f6784c-9ptdj\" (UID: \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\") " pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 16:58:22.837533 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.837366    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-dshm\") pod \"auth-test-model-8269b5-kserve-95f6784c-9ptdj\" (UID: \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\") " pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 16:58:22.837533 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.837404    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkt9l\" (UniqueName: \"kubernetes.io/projected/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-kube-api-access-pkt9l\") pod \"auth-test-model-8269b5-kserve-95f6784c-9ptdj\" (UID: \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\") " pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 16:58:22.837750 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.837707    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-tmp-dir\") pod \"auth-test-model-8269b5-kserve-95f6784c-9ptdj\" (UID: \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\") " pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 16:58:22.837859 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.837776    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-model-cache\") pod \"auth-test-model-8269b5-kserve-95f6784c-9ptdj\" (UID: \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\") " pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 16:58:22.837859 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.837821    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-home\") pod \"auth-test-model-8269b5-kserve-95f6784c-9ptdj\" (UID: \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\") " pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 16:58:22.839696 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.839671    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-dshm\") pod \"auth-test-model-8269b5-kserve-95f6784c-9ptdj\" (UID: \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\") " pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 16:58:22.839897 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.839878    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-tls-certs\") pod \"auth-test-model-8269b5-kserve-95f6784c-9ptdj\" (UID: \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\") " pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 16:58:22.846191 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.846163    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkt9l\" (UniqueName: \"kubernetes.io/projected/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-kube-api-access-pkt9l\") pod \"auth-test-model-8269b5-kserve-95f6784c-9ptdj\" (UID: \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\") " pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 16:58:22.941198 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:22.941096    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 16:58:23.083614 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:23.083588    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"]
Jul 15 16:58:23.085410 ip-10-0-134-132 kubenswrapper[2580]: W0715 16:58:23.085386    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod265fa92a_b8f3_4b7b_90ea_a983ce754cb2.slice/crio-d2c265cf6afe8444287e54f8c8dcb6b51301736f80af2d927f60174de9c565f8 WatchSource:0}: Error finding container d2c265cf6afe8444287e54f8c8dcb6b51301736f80af2d927f60174de9c565f8: Status 404 returned error can't find the container with id d2c265cf6afe8444287e54f8c8dcb6b51301736f80af2d927f60174de9c565f8
Jul 15 16:58:23.272600 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:23.272511    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj" event={"ID":"265fa92a-b8f3-4b7b-90ea-a983ce754cb2","Type":"ContainerStarted","Data":"b0a6756f10ab80718e714b6415d41ce0847a47245245ab09bc544562c11be036"}
Jul 15 16:58:23.272600 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:23.272558    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj" event={"ID":"265fa92a-b8f3-4b7b-90ea-a983ce754cb2","Type":"ContainerStarted","Data":"d2c265cf6afe8444287e54f8c8dcb6b51301736f80af2d927f60174de9c565f8"}
Jul 15 16:58:23.291553 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:23.291500    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj" podStartSLOduration=1.291484275 podStartE2EDuration="1.291484275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:58:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:58:23.290920051 +0000 UTC m=+2504.937960614" watchObservedRunningTime="2026-07-15 16:58:23.291484275 +0000 UTC m=+2504.938524811"
Jul 15 16:58:25.790963 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:25.790926    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:58:26.580454 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:26.580421    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:58:26.722986 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:26.722951    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:58:32.942288 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:32.942192    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 16:58:32.942288 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:32.942250    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 16:58:32.954727 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:32.954701    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 16:58:33.335304 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:33.335269    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 16:58:48.788442 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:48.788405    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:58:52.093800 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:58:52.093749    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:59:06.881958 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:59:06.881923    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:59:25.305004 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:59:25.304960    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:59:40.595732 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:59:40.595692    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 16:59:50.854482 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:59:50.854440    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 16:59:54.894991 ip-10-0-134-132 kubenswrapper[2580]: I0715 16:59:54.894958    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 17:00:00.193739 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:00.193653    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-021cd5-29735580-ddzvx"]
Jul 15 17:00:00.197114 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:00.197095    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735580-jx74m"]
Jul 15 17:00:00.197228 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:00.197208    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-021cd5-29735580-ddzvx"
Jul 15 17:00:00.200442 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:00.200420    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8269b5-29735580-8dq2g"]
Jul 15 17:00:00.200549 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:00.200510    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735580-jx74m"
Jul 15 17:00:00.203064 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:00.203039    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lsp65\""
Jul 15 17:00:00.204122 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:00.204104    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8269b5-29735580-8dq2g"
Jul 15 17:00:00.208300 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:00.208284    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-021cd5-29735580-ddzvx"
Jul 15 17:00:00.215953 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:00.215936    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735580-jx74m"
Jul 15 17:00:00.237520 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:00.237459    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-021cd5-29735580-ddzvx"]
Jul 15 17:00:00.239668 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:00.239617    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735580-jx74m"]
Jul 15 17:00:00.242440 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:00.242413    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8269b5-29735580-8dq2g"]
Jul 15 17:00:00.244988 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:00.244968    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8269b5-29735580-8dq2g"
Jul 15 17:00:00.410359 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:00.410326    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-021cd5-29735580-ddzvx"]
Jul 15 17:00:00.412699 ip-10-0-134-132 kubenswrapper[2580]: W0715 17:00:00.412671    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72e50f4b_5a83_4c1c_9d32_03b8041cf57c.slice/crio-f61d5882f11be4c94f106151f3e3d7aa8e12cde129ce6584288a67c96af1911f WatchSource:0}: Error finding container f61d5882f11be4c94f106151f3e3d7aa8e12cde129ce6584288a67c96af1911f: Status 404 returned error can't find the container with id f61d5882f11be4c94f106151f3e3d7aa8e12cde129ce6584288a67c96af1911f
Jul 15 17:00:00.432322 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:00.432295    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735580-jx74m"]
Jul 15 17:00:00.451623 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:00.451280    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8269b5-29735580-8dq2g"]
Jul 15 17:00:00.454524 ip-10-0-134-132 kubenswrapper[2580]: W0715 17:00:00.454494    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod329af7e1_f343_4973_a46a_0ae1495a58b3.slice/crio-25e10708807049b6b4998182fa6022dcbbf873d7348d7277004d6d14df1cd6bb WatchSource:0}: Error finding container 25e10708807049b6b4998182fa6022dcbbf873d7348d7277004d6d14df1cd6bb: Status 404 returned error can't find the container with id 25e10708807049b6b4998182fa6022dcbbf873d7348d7277004d6d14df1cd6bb
Jul 15 17:00:00.772047 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:00.772003    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735580-jx74m" event={"ID":"329af7e1-f343-4973-a46a-0ae1495a58b3","Type":"ContainerStarted","Data":"25e10708807049b6b4998182fa6022dcbbf873d7348d7277004d6d14df1cd6bb"}
Jul 15 17:00:00.773516 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:00.773489    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-021cd5-29735580-ddzvx" event={"ID":"72e50f4b-5a83-4c1c-9d32-03b8041cf57c","Type":"ContainerStarted","Data":"9d8f13ec22c9723aca251e717183d443e983570a0163de044a67a0cabe59aac1"}
Jul 15 17:00:00.773648 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:00.773539    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-021cd5-29735580-ddzvx" event={"ID":"72e50f4b-5a83-4c1c-9d32-03b8041cf57c","Type":"ContainerStarted","Data":"f61d5882f11be4c94f106151f3e3d7aa8e12cde129ce6584288a67c96af1911f"}
Jul 15 17:00:00.775051 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:00.775029    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8269b5-29735580-8dq2g" event={"ID":"793c123e-c10a-4dee-8bd5-edd191b14007","Type":"ContainerStarted","Data":"d1427047d90f4a9df56d8084fffb6dac5000c0b1dbed6f2bfe1d8c5cf8bb3274"}
Jul 15 17:00:00.775106 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:00.775074    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8269b5-29735580-8dq2g" event={"ID":"793c123e-c10a-4dee-8bd5-edd191b14007","Type":"ContainerStarted","Data":"5843c4336cc1731a1447921a1440d05c0835df7c00e8afa095e6f7d5ab60c09f"}
Jul 15 17:00:00.801064 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:00.801016    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-021cd5-29735580-ddzvx" podStartSLOduration=0.801001575 podStartE2EDuration="801.001575ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:00.800044881 +0000 UTC m=+2602.447085416" watchObservedRunningTime="2026-07-15 17:00:00.801001575 +0000 UTC m=+2602.448042180"
Jul 15 17:00:00.841808 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:00.841737    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8269b5-29735580-8dq2g" podStartSLOduration=0.841721828 podStartE2EDuration="841.721828ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:00.841481021 +0000 UTC m=+2602.488521556" watchObservedRunningTime="2026-07-15 17:00:00.841721828 +0000 UTC m=+2602.488762363"
Jul 15 17:00:01.782626 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:01.782534    2580 generic.go:356] "Generic (PLEG): container finished" podID="329af7e1-f343-4973-a46a-0ae1495a58b3" containerID="976037adb79e6ae4ca0660669d7c0dcf1a7bf2e927fbc32477d4f74a29b20878" exitCode=0
Jul 15 17:00:01.782626 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:01.782608    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735580-jx74m" event={"ID":"329af7e1-f343-4973-a46a-0ae1495a58b3","Type":"ContainerDied","Data":"976037adb79e6ae4ca0660669d7c0dcf1a7bf2e927fbc32477d4f74a29b20878"}
Jul 15 17:00:01.784208 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:01.784176    2580 generic.go:356] "Generic (PLEG): container finished" podID="72e50f4b-5a83-4c1c-9d32-03b8041cf57c" containerID="9d8f13ec22c9723aca251e717183d443e983570a0163de044a67a0cabe59aac1" exitCode=0
Jul 15 17:00:01.784323 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:01.784261    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-021cd5-29735580-ddzvx" event={"ID":"72e50f4b-5a83-4c1c-9d32-03b8041cf57c","Type":"ContainerDied","Data":"9d8f13ec22c9723aca251e717183d443e983570a0163de044a67a0cabe59aac1"}
Jul 15 17:00:01.785822 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:01.785802    2580 generic.go:356] "Generic (PLEG): container finished" podID="793c123e-c10a-4dee-8bd5-edd191b14007" containerID="d1427047d90f4a9df56d8084fffb6dac5000c0b1dbed6f2bfe1d8c5cf8bb3274" exitCode=0
Jul 15 17:00:01.785918 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:01.785835    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8269b5-29735580-8dq2g" event={"ID":"793c123e-c10a-4dee-8bd5-edd191b14007","Type":"ContainerDied","Data":"d1427047d90f4a9df56d8084fffb6dac5000c0b1dbed6f2bfe1d8c5cf8bb3274"}
Jul 15 17:00:02.965098 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:02.965071    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8269b5-29735580-8dq2g"
Jul 15 17:00:02.968072 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:02.968053    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735580-jx74m"
Jul 15 17:00:02.974555 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:02.974537    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-021cd5-29735580-ddzvx"
Jul 15 17:00:03.797964 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:03.797934    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735580-jx74m"
Jul 15 17:00:03.798153 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:03.797935    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735580-jx74m" event={"ID":"329af7e1-f343-4973-a46a-0ae1495a58b3","Type":"ContainerDied","Data":"25e10708807049b6b4998182fa6022dcbbf873d7348d7277004d6d14df1cd6bb"}
Jul 15 17:00:03.798153 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:03.798051    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="25e10708807049b6b4998182fa6022dcbbf873d7348d7277004d6d14df1cd6bb"
Jul 15 17:00:03.799322 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:03.799300    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-021cd5-29735580-ddzvx"
Jul 15 17:00:03.799465 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:03.799350    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-021cd5-29735580-ddzvx" event={"ID":"72e50f4b-5a83-4c1c-9d32-03b8041cf57c","Type":"ContainerDied","Data":"f61d5882f11be4c94f106151f3e3d7aa8e12cde129ce6584288a67c96af1911f"}
Jul 15 17:00:03.799465 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:03.799397    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f61d5882f11be4c94f106151f3e3d7aa8e12cde129ce6584288a67c96af1911f"
Jul 15 17:00:03.800788 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:03.800772    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8269b5-29735580-8dq2g"
Jul 15 17:00:03.800890 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:03.800789    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8269b5-29735580-8dq2g" event={"ID":"793c123e-c10a-4dee-8bd5-edd191b14007","Type":"ContainerDied","Data":"5843c4336cc1731a1447921a1440d05c0835df7c00e8afa095e6f7d5ab60c09f"}
Jul 15 17:00:03.800890 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:03.800823    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5843c4336cc1731a1447921a1440d05c0835df7c00e8afa095e6f7d5ab60c09f"
Jul 15 17:00:12.242527 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:12.242490    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 17:00:25.883231 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:25.883197    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:00:25.987388 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:25.987347    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:00:31.089436 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:31.089397    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 17:00:42.325928 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:42.325889    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"]
Jul 15 17:00:42.355399 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:42.355368    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:00:43.284977 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:43.284938    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:00:47.367992 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:47.367947    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 17:00:52.621951 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.621914    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"]
Jul 15 17:00:52.622499 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.622482    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72e50f4b-5a83-4c1c-9d32-03b8041cf57c" containerName="cleanup"
Jul 15 17:00:52.622547 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.622502    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="72e50f4b-5a83-4c1c-9d32-03b8041cf57c" containerName="cleanup"
Jul 15 17:00:52.622547 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.622529    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="329af7e1-f343-4973-a46a-0ae1495a58b3" containerName="cleanup"
Jul 15 17:00:52.622547 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.622537    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="329af7e1-f343-4973-a46a-0ae1495a58b3" containerName="cleanup"
Jul 15 17:00:52.622646 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.622556    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="793c123e-c10a-4dee-8bd5-edd191b14007" containerName="cleanup"
Jul 15 17:00:52.622646 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.622566    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="793c123e-c10a-4dee-8bd5-edd191b14007" containerName="cleanup"
Jul 15 17:00:52.622646 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.622633    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="72e50f4b-5a83-4c1c-9d32-03b8041cf57c" containerName="cleanup"
Jul 15 17:00:52.622646 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.622643    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="793c123e-c10a-4dee-8bd5-edd191b14007" containerName="cleanup"
Jul 15 17:00:52.622782 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.622652    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="329af7e1-f343-4973-a46a-0ae1495a58b3" containerName="cleanup"
Jul 15 17:00:52.626264 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.626244    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:00:52.628931 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.628908    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8269b5\"/\"sub-test-model-8269b5-kserve-self-signed-certs\""
Jul 15 17:00:52.640355 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.640328    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"]
Jul 15 17:00:52.662617 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.662578    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:00:52.673741 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.673716    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f4bebec-f960-4df0-8ccf-99b41efe2989-tls-certs\") pod \"sub-test-model-8269b5-kserve-55cb6995f5-8p2v8\" (UID: \"9f4bebec-f960-4df0-8ccf-99b41efe2989\") " pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:00:52.673861 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.673774    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f4bebec-f960-4df0-8ccf-99b41efe2989-tmp-dir\") pod \"sub-test-model-8269b5-kserve-55cb6995f5-8p2v8\" (UID: \"9f4bebec-f960-4df0-8ccf-99b41efe2989\") " pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:00:52.673861 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.673830    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f4bebec-f960-4df0-8ccf-99b41efe2989-dshm\") pod \"sub-test-model-8269b5-kserve-55cb6995f5-8p2v8\" (UID: \"9f4bebec-f960-4df0-8ccf-99b41efe2989\") " pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:00:52.673934 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.673882    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f4bebec-f960-4df0-8ccf-99b41efe2989-model-cache\") pod \"sub-test-model-8269b5-kserve-55cb6995f5-8p2v8\" (UID: \"9f4bebec-f960-4df0-8ccf-99b41efe2989\") " pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:00:52.673934 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.673900    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbq8n\" (UniqueName: \"kubernetes.io/projected/9f4bebec-f960-4df0-8ccf-99b41efe2989-kube-api-access-nbq8n\") pod \"sub-test-model-8269b5-kserve-55cb6995f5-8p2v8\" (UID: \"9f4bebec-f960-4df0-8ccf-99b41efe2989\") " pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:00:52.673999 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.673938    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f4bebec-f960-4df0-8ccf-99b41efe2989-home\") pod \"sub-test-model-8269b5-kserve-55cb6995f5-8p2v8\" (UID: \"9f4bebec-f960-4df0-8ccf-99b41efe2989\") " pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:00:52.774880 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.774844    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f4bebec-f960-4df0-8ccf-99b41efe2989-tls-certs\") pod \"sub-test-model-8269b5-kserve-55cb6995f5-8p2v8\" (UID: \"9f4bebec-f960-4df0-8ccf-99b41efe2989\") " pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:00:52.775035 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.774901    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f4bebec-f960-4df0-8ccf-99b41efe2989-tmp-dir\") pod \"sub-test-model-8269b5-kserve-55cb6995f5-8p2v8\" (UID: \"9f4bebec-f960-4df0-8ccf-99b41efe2989\") " pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:00:52.775109 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.775023    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f4bebec-f960-4df0-8ccf-99b41efe2989-dshm\") pod \"sub-test-model-8269b5-kserve-55cb6995f5-8p2v8\" (UID: \"9f4bebec-f960-4df0-8ccf-99b41efe2989\") " pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:00:52.775109 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.775072    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f4bebec-f960-4df0-8ccf-99b41efe2989-model-cache\") pod \"sub-test-model-8269b5-kserve-55cb6995f5-8p2v8\" (UID: \"9f4bebec-f960-4df0-8ccf-99b41efe2989\") " pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:00:52.775224 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.775113    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbq8n\" (UniqueName: \"kubernetes.io/projected/9f4bebec-f960-4df0-8ccf-99b41efe2989-kube-api-access-nbq8n\") pod \"sub-test-model-8269b5-kserve-55cb6995f5-8p2v8\" (UID: \"9f4bebec-f960-4df0-8ccf-99b41efe2989\") " pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:00:52.775282 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.775263    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f4bebec-f960-4df0-8ccf-99b41efe2989-home\") pod \"sub-test-model-8269b5-kserve-55cb6995f5-8p2v8\" (UID: \"9f4bebec-f960-4df0-8ccf-99b41efe2989\") " pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:00:52.775452 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.775432    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f4bebec-f960-4df0-8ccf-99b41efe2989-tmp-dir\") pod \"sub-test-model-8269b5-kserve-55cb6995f5-8p2v8\" (UID: \"9f4bebec-f960-4df0-8ccf-99b41efe2989\") " pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:00:52.775537 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.775437    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f4bebec-f960-4df0-8ccf-99b41efe2989-model-cache\") pod \"sub-test-model-8269b5-kserve-55cb6995f5-8p2v8\" (UID: \"9f4bebec-f960-4df0-8ccf-99b41efe2989\") " pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:00:52.775644 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.775624    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f4bebec-f960-4df0-8ccf-99b41efe2989-home\") pod \"sub-test-model-8269b5-kserve-55cb6995f5-8p2v8\" (UID: \"9f4bebec-f960-4df0-8ccf-99b41efe2989\") " pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:00:52.777771 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.777703    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f4bebec-f960-4df0-8ccf-99b41efe2989-dshm\") pod \"sub-test-model-8269b5-kserve-55cb6995f5-8p2v8\" (UID: \"9f4bebec-f960-4df0-8ccf-99b41efe2989\") " pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:00:52.778171 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.778144    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f4bebec-f960-4df0-8ccf-99b41efe2989-tls-certs\") pod \"sub-test-model-8269b5-kserve-55cb6995f5-8p2v8\" (UID: \"9f4bebec-f960-4df0-8ccf-99b41efe2989\") " pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:00:52.783733 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.783714    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbq8n\" (UniqueName: \"kubernetes.io/projected/9f4bebec-f960-4df0-8ccf-99b41efe2989-kube-api-access-nbq8n\") pod \"sub-test-model-8269b5-kserve-55cb6995f5-8p2v8\" (UID: \"9f4bebec-f960-4df0-8ccf-99b41efe2989\") " pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:00:52.935859 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:52.935771    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:00:53.066543 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:53.066517    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"]
Jul 15 17:00:53.068505 ip-10-0-134-132 kubenswrapper[2580]: W0715 17:00:53.068478    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f4bebec_f960_4df0_8ccf_99b41efe2989.slice/crio-f713680e9c2ca546d05fa771a1eb2852ecfcadbb4cb87e79525a979cebc20f21 WatchSource:0}: Error finding container f713680e9c2ca546d05fa771a1eb2852ecfcadbb4cb87e79525a979cebc20f21: Status 404 returned error can't find the container with id f713680e9c2ca546d05fa771a1eb2852ecfcadbb4cb87e79525a979cebc20f21
Jul 15 17:00:54.066190 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:54.066153    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8" event={"ID":"9f4bebec-f960-4df0-8ccf-99b41efe2989","Type":"ContainerStarted","Data":"563a2f02b3045fb803c131dc6062536ad73d4f1bc7d4b0163c167f01af974525"}
Jul 15 17:00:54.066190 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:54.066202    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8" event={"ID":"9f4bebec-f960-4df0-8ccf-99b41efe2989","Type":"ContainerStarted","Data":"f713680e9c2ca546d05fa771a1eb2852ecfcadbb4cb87e79525a979cebc20f21"}
Jul 15 17:00:54.086260 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:54.086201    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8" podStartSLOduration=2.086185578 podStartE2EDuration="2.086185578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:00:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:54.08569154 +0000 UTC m=+2655.732732102" watchObservedRunningTime="2026-07-15 17:00:54.086185578 +0000 UTC m=+2655.733226120"
Jul 15 17:00:57.345967 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:57.345897    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj" podUID="265fa92a-b8f3-4b7b-90ea-a983ce754cb2" containerName="main" containerID="cri-o://b0a6756f10ab80718e714b6415d41ce0847a47245245ab09bc544562c11be036" gracePeriod=45
Jul 15 17:00:57.600435 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:57.600373    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 17:00:57.716650 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:57.716621    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-model-cache\") pod \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\" (UID: \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\") "
Jul 15 17:00:57.716839 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:57.716668    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-dshm\") pod \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\" (UID: \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\") "
Jul 15 17:00:57.716839 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:57.716711    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-tmp-dir\") pod \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\" (UID: \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\") "
Jul 15 17:00:57.716839 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:57.716749    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-home\") pod \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\" (UID: \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\") "
Jul 15 17:00:57.716839 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:57.716820    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkt9l\" (UniqueName: \"kubernetes.io/projected/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-kube-api-access-pkt9l\") pod \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\" (UID: \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\") "
Jul 15 17:00:57.717068 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:57.716851    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-tls-certs\") pod \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\" (UID: \"265fa92a-b8f3-4b7b-90ea-a983ce754cb2\") "
Jul 15 17:00:57.717068 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:57.716955    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-model-cache" (OuterVolumeSpecName: "model-cache") pod "265fa92a-b8f3-4b7b-90ea-a983ce754cb2" (UID: "265fa92a-b8f3-4b7b-90ea-a983ce754cb2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:00:57.717068 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:57.716995    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "265fa92a-b8f3-4b7b-90ea-a983ce754cb2" (UID: "265fa92a-b8f3-4b7b-90ea-a983ce754cb2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:00:57.717202 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:57.717115    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-home" (OuterVolumeSpecName: "home") pod "265fa92a-b8f3-4b7b-90ea-a983ce754cb2" (UID: "265fa92a-b8f3-4b7b-90ea-a983ce754cb2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:00:57.717202 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:57.717185    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-tmp-dir\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:00:57.717273 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:57.717208    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-model-cache\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:00:57.718899 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:57.718879    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-dshm" (OuterVolumeSpecName: "dshm") pod "265fa92a-b8f3-4b7b-90ea-a983ce754cb2" (UID: "265fa92a-b8f3-4b7b-90ea-a983ce754cb2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:00:57.718958 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:57.718936    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "265fa92a-b8f3-4b7b-90ea-a983ce754cb2" (UID: "265fa92a-b8f3-4b7b-90ea-a983ce754cb2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:00:57.719042 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:57.719021    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-kube-api-access-pkt9l" (OuterVolumeSpecName: "kube-api-access-pkt9l") pod "265fa92a-b8f3-4b7b-90ea-a983ce754cb2" (UID: "265fa92a-b8f3-4b7b-90ea-a983ce754cb2"). InnerVolumeSpecName "kube-api-access-pkt9l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:00:57.818203 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:57.818176    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkt9l\" (UniqueName: \"kubernetes.io/projected/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-kube-api-access-pkt9l\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:00:57.818203 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:57.818201    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-tls-certs\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:00:57.818367 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:57.818210    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-dshm\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:00:57.818367 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:57.818219    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/265fa92a-b8f3-4b7b-90ea-a983ce754cb2-home\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:00:58.088912 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:58.088878    2580 generic.go:356] "Generic (PLEG): container finished" podID="265fa92a-b8f3-4b7b-90ea-a983ce754cb2" containerID="b0a6756f10ab80718e714b6415d41ce0847a47245245ab09bc544562c11be036" exitCode=0
Jul 15 17:00:58.089099 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:58.088920    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"
Jul 15 17:00:58.089099 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:58.088971    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj" event={"ID":"265fa92a-b8f3-4b7b-90ea-a983ce754cb2","Type":"ContainerDied","Data":"b0a6756f10ab80718e714b6415d41ce0847a47245245ab09bc544562c11be036"}
Jul 15 17:00:58.089099 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:58.089016    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj" event={"ID":"265fa92a-b8f3-4b7b-90ea-a983ce754cb2","Type":"ContainerDied","Data":"d2c265cf6afe8444287e54f8c8dcb6b51301736f80af2d927f60174de9c565f8"}
Jul 15 17:00:58.089099 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:58.089035    2580 scope.go:117] "RemoveContainer" containerID="b0a6756f10ab80718e714b6415d41ce0847a47245245ab09bc544562c11be036"
Jul 15 17:00:58.102299 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:58.102279    2580 scope.go:117] "RemoveContainer" containerID="b0a6756f10ab80718e714b6415d41ce0847a47245245ab09bc544562c11be036"
Jul 15 17:00:58.102562 ip-10-0-134-132 kubenswrapper[2580]: E0715 17:00:58.102537    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0a6756f10ab80718e714b6415d41ce0847a47245245ab09bc544562c11be036\": container with ID starting with b0a6756f10ab80718e714b6415d41ce0847a47245245ab09bc544562c11be036 not found: ID does not exist" containerID="b0a6756f10ab80718e714b6415d41ce0847a47245245ab09bc544562c11be036"
Jul 15 17:00:58.102665 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:58.102569    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0a6756f10ab80718e714b6415d41ce0847a47245245ab09bc544562c11be036"} err="failed to get container status \"b0a6756f10ab80718e714b6415d41ce0847a47245245ab09bc544562c11be036\": rpc error: code = NotFound desc = could not find container \"b0a6756f10ab80718e714b6415d41ce0847a47245245ab09bc544562c11be036\": container with ID starting with b0a6756f10ab80718e714b6415d41ce0847a47245245ab09bc544562c11be036 not found: ID does not exist"
Jul 15 17:00:58.115154 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:58.115130    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"]
Jul 15 17:00:58.122081 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:58.122056    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-8269b5/auth-test-model-8269b5-kserve-95f6784c-9ptdj"]
Jul 15 17:00:58.880854 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:00:58.880819    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="265fa92a-b8f3-4b7b-90ea-a983ce754cb2" path="/var/lib/kubelet/pods/265fa92a-b8f3-4b7b-90ea-a983ce754cb2/volumes"
Jul 15 17:01:01.178620 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:01.178585    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 17:01:02.936840 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:02.936806    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:01:02.937228 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:02.936856    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:01:02.949451 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:02.949419    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:01:03.126073 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:03.126045    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:01:17.876057 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:17.876018    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:01:20.532077 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:20.532022    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 17:01:35.588232 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:35.588149    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 17:01:38.946015 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:38.945975    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-295d8_46356463-bfb1-42b8-8871-71357489a95b/ovn-acl-logging/0.log"
Jul 15 17:01:38.952091 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:38.952069    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-295d8_46356463-bfb1-42b8-8871-71357489a95b/ovn-acl-logging/0.log"
Jul 15 17:01:40.771418 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:40.771383    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:01:41.001427 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:41.001389    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:01:50.461322 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:50.461285    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:01:52.412627 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:52.412581    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"]
Jul 15 17:01:52.426732 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:52.426698    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:01:53.252171 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:53.252133    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:01:57.797713 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:57.797677    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 17:01:58.931523 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:58.931489    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"]
Jul 15 17:01:58.932342 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:58.932318    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="265fa92a-b8f3-4b7b-90ea-a983ce754cb2" containerName="main"
Jul 15 17:01:58.932342 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:58.932343    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="265fa92a-b8f3-4b7b-90ea-a983ce754cb2" containerName="main"
Jul 15 17:01:58.932502 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:58.932449    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="265fa92a-b8f3-4b7b-90ea-a983ce754cb2" containerName="main"
Jul 15 17:01:58.935943 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:58.935916    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:01:58.938539 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:58.938519    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8269b5\"/\"rate-test-model-8269b5-kserve-self-signed-certs\""
Jul 15 17:01:58.944927 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:58.944906    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"]
Jul 15 17:01:58.966721 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:58.966668    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:01:59.071095 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:59.071051    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db3880df-d8cc-4759-b567-baf9cf57eab1-tmp-dir\") pod \"rate-test-model-8269b5-kserve-6fdf68f79-fbk7q\" (UID: \"db3880df-d8cc-4759-b567-baf9cf57eab1\") " pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:01:59.071095 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:59.071094    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db3880df-d8cc-4759-b567-baf9cf57eab1-dshm\") pod \"rate-test-model-8269b5-kserve-6fdf68f79-fbk7q\" (UID: \"db3880df-d8cc-4759-b567-baf9cf57eab1\") " pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:01:59.071306 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:59.071122    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2gkx\" (UniqueName: \"kubernetes.io/projected/db3880df-d8cc-4759-b567-baf9cf57eab1-kube-api-access-f2gkx\") pod \"rate-test-model-8269b5-kserve-6fdf68f79-fbk7q\" (UID: \"db3880df-d8cc-4759-b567-baf9cf57eab1\") " pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:01:59.071306 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:59.071205    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db3880df-d8cc-4759-b567-baf9cf57eab1-home\") pod \"rate-test-model-8269b5-kserve-6fdf68f79-fbk7q\" (UID: \"db3880df-d8cc-4759-b567-baf9cf57eab1\") " pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:01:59.071306 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:59.071253    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db3880df-d8cc-4759-b567-baf9cf57eab1-tls-certs\") pod \"rate-test-model-8269b5-kserve-6fdf68f79-fbk7q\" (UID: \"db3880df-d8cc-4759-b567-baf9cf57eab1\") " pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:01:59.071406 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:59.071314    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db3880df-d8cc-4759-b567-baf9cf57eab1-model-cache\") pod \"rate-test-model-8269b5-kserve-6fdf68f79-fbk7q\" (UID: \"db3880df-d8cc-4759-b567-baf9cf57eab1\") " pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:01:59.172162 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:59.172121    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db3880df-d8cc-4759-b567-baf9cf57eab1-model-cache\") pod \"rate-test-model-8269b5-kserve-6fdf68f79-fbk7q\" (UID: \"db3880df-d8cc-4759-b567-baf9cf57eab1\") " pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:01:59.172341 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:59.172170    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db3880df-d8cc-4759-b567-baf9cf57eab1-tmp-dir\") pod \"rate-test-model-8269b5-kserve-6fdf68f79-fbk7q\" (UID: \"db3880df-d8cc-4759-b567-baf9cf57eab1\") " pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:01:59.172341 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:59.172289    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db3880df-d8cc-4759-b567-baf9cf57eab1-dshm\") pod \"rate-test-model-8269b5-kserve-6fdf68f79-fbk7q\" (UID: \"db3880df-d8cc-4759-b567-baf9cf57eab1\") " pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:01:59.172427 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:59.172343    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2gkx\" (UniqueName: \"kubernetes.io/projected/db3880df-d8cc-4759-b567-baf9cf57eab1-kube-api-access-f2gkx\") pod \"rate-test-model-8269b5-kserve-6fdf68f79-fbk7q\" (UID: \"db3880df-d8cc-4759-b567-baf9cf57eab1\") " pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:01:59.172427 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:59.172398    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db3880df-d8cc-4759-b567-baf9cf57eab1-home\") pod \"rate-test-model-8269b5-kserve-6fdf68f79-fbk7q\" (UID: \"db3880df-d8cc-4759-b567-baf9cf57eab1\") " pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:01:59.172527 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:59.172444    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db3880df-d8cc-4759-b567-baf9cf57eab1-tls-certs\") pod \"rate-test-model-8269b5-kserve-6fdf68f79-fbk7q\" (UID: \"db3880df-d8cc-4759-b567-baf9cf57eab1\") " pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:01:59.172527 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:59.172510    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db3880df-d8cc-4759-b567-baf9cf57eab1-tmp-dir\") pod \"rate-test-model-8269b5-kserve-6fdf68f79-fbk7q\" (UID: \"db3880df-d8cc-4759-b567-baf9cf57eab1\") " pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:01:59.172639 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:59.172558    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db3880df-d8cc-4759-b567-baf9cf57eab1-model-cache\") pod \"rate-test-model-8269b5-kserve-6fdf68f79-fbk7q\" (UID: \"db3880df-d8cc-4759-b567-baf9cf57eab1\") " pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:01:59.172817 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:59.172797    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db3880df-d8cc-4759-b567-baf9cf57eab1-home\") pod \"rate-test-model-8269b5-kserve-6fdf68f79-fbk7q\" (UID: \"db3880df-d8cc-4759-b567-baf9cf57eab1\") " pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:01:59.174491 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:59.174472    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db3880df-d8cc-4759-b567-baf9cf57eab1-dshm\") pod \"rate-test-model-8269b5-kserve-6fdf68f79-fbk7q\" (UID: \"db3880df-d8cc-4759-b567-baf9cf57eab1\") " pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:01:59.174875 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:59.174856    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db3880df-d8cc-4759-b567-baf9cf57eab1-tls-certs\") pod \"rate-test-model-8269b5-kserve-6fdf68f79-fbk7q\" (UID: \"db3880df-d8cc-4759-b567-baf9cf57eab1\") " pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:01:59.182539 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:59.182478    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2gkx\" (UniqueName: \"kubernetes.io/projected/db3880df-d8cc-4759-b567-baf9cf57eab1-kube-api-access-f2gkx\") pod \"rate-test-model-8269b5-kserve-6fdf68f79-fbk7q\" (UID: \"db3880df-d8cc-4759-b567-baf9cf57eab1\") " pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:01:59.246369 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:59.246333    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:01:59.389413 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:59.389368    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"]
Jul 15 17:01:59.391894 ip-10-0-134-132 kubenswrapper[2580]: W0715 17:01:59.391862    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb3880df_d8cc_4759_b567_baf9cf57eab1.slice/crio-239a3a373e9bb04b37cf6a811a3b5a6037cebe29c649fccb72a50c242ccf603a WatchSource:0}: Error finding container 239a3a373e9bb04b37cf6a811a3b5a6037cebe29c649fccb72a50c242ccf603a: Status 404 returned error can't find the container with id 239a3a373e9bb04b37cf6a811a3b5a6037cebe29c649fccb72a50c242ccf603a
Jul 15 17:01:59.407077 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:01:59.407047    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q" event={"ID":"db3880df-d8cc-4759-b567-baf9cf57eab1","Type":"ContainerStarted","Data":"239a3a373e9bb04b37cf6a811a3b5a6037cebe29c649fccb72a50c242ccf603a"}
Jul 15 17:02:00.413233 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:00.413197    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q" event={"ID":"db3880df-d8cc-4759-b567-baf9cf57eab1","Type":"ContainerStarted","Data":"ce347657d4e1d7dd240895509c1a9980ce61c0827e054a147396a73c9eb96fd0"}
Jul 15 17:02:00.431326 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:00.431269    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q" podStartSLOduration=2.431254079 podStartE2EDuration="2.431254079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:01:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:02:00.43013009 +0000 UTC m=+2722.077170627" watchObservedRunningTime="2026-07-15 17:02:00.431254079 +0000 UTC m=+2722.078294614"
Jul 15 17:02:07.433468 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:07.433414    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8" podUID="9f4bebec-f960-4df0-8ccf-99b41efe2989" containerName="main" containerID="cri-o://563a2f02b3045fb803c131dc6062536ad73d4f1bc7d4b0163c167f01af974525" gracePeriod=45
Jul 15 17:02:07.689940 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:07.689880    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:02:07.850862 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:07.850826    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f4bebec-f960-4df0-8ccf-99b41efe2989-dshm\") pod \"9f4bebec-f960-4df0-8ccf-99b41efe2989\" (UID: \"9f4bebec-f960-4df0-8ccf-99b41efe2989\") "
Jul 15 17:02:07.850862 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:07.850869    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbq8n\" (UniqueName: \"kubernetes.io/projected/9f4bebec-f960-4df0-8ccf-99b41efe2989-kube-api-access-nbq8n\") pod \"9f4bebec-f960-4df0-8ccf-99b41efe2989\" (UID: \"9f4bebec-f960-4df0-8ccf-99b41efe2989\") "
Jul 15 17:02:07.851113 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:07.850891    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f4bebec-f960-4df0-8ccf-99b41efe2989-tls-certs\") pod \"9f4bebec-f960-4df0-8ccf-99b41efe2989\" (UID: \"9f4bebec-f960-4df0-8ccf-99b41efe2989\") "
Jul 15 17:02:07.851113 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:07.850946    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f4bebec-f960-4df0-8ccf-99b41efe2989-tmp-dir\") pod \"9f4bebec-f960-4df0-8ccf-99b41efe2989\" (UID: \"9f4bebec-f960-4df0-8ccf-99b41efe2989\") "
Jul 15 17:02:07.851113 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:07.850973    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f4bebec-f960-4df0-8ccf-99b41efe2989-home\") pod \"9f4bebec-f960-4df0-8ccf-99b41efe2989\" (UID: \"9f4bebec-f960-4df0-8ccf-99b41efe2989\") "
Jul 15 17:02:07.851113 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:07.851013    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f4bebec-f960-4df0-8ccf-99b41efe2989-model-cache\") pod \"9f4bebec-f960-4df0-8ccf-99b41efe2989\" (UID: \"9f4bebec-f960-4df0-8ccf-99b41efe2989\") "
Jul 15 17:02:07.851323 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:07.851263    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f4bebec-f960-4df0-8ccf-99b41efe2989-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9f4bebec-f960-4df0-8ccf-99b41efe2989" (UID: "9f4bebec-f960-4df0-8ccf-99b41efe2989"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:02:07.851385 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:07.851306    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f4bebec-f960-4df0-8ccf-99b41efe2989-home" (OuterVolumeSpecName: "home") pod "9f4bebec-f960-4df0-8ccf-99b41efe2989" (UID: "9f4bebec-f960-4df0-8ccf-99b41efe2989"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:02:07.851485 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:07.851461    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f4bebec-f960-4df0-8ccf-99b41efe2989-model-cache" (OuterVolumeSpecName: "model-cache") pod "9f4bebec-f960-4df0-8ccf-99b41efe2989" (UID: "9f4bebec-f960-4df0-8ccf-99b41efe2989"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:02:07.853087 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:07.853064    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f4bebec-f960-4df0-8ccf-99b41efe2989-dshm" (OuterVolumeSpecName: "dshm") pod "9f4bebec-f960-4df0-8ccf-99b41efe2989" (UID: "9f4bebec-f960-4df0-8ccf-99b41efe2989"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:02:07.853213 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:07.853119    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f4bebec-f960-4df0-8ccf-99b41efe2989-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9f4bebec-f960-4df0-8ccf-99b41efe2989" (UID: "9f4bebec-f960-4df0-8ccf-99b41efe2989"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:02:07.853213 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:07.853156    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f4bebec-f960-4df0-8ccf-99b41efe2989-kube-api-access-nbq8n" (OuterVolumeSpecName: "kube-api-access-nbq8n") pod "9f4bebec-f960-4df0-8ccf-99b41efe2989" (UID: "9f4bebec-f960-4df0-8ccf-99b41efe2989"). InnerVolumeSpecName "kube-api-access-nbq8n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:02:07.952308 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:07.952207    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f4bebec-f960-4df0-8ccf-99b41efe2989-dshm\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:02:07.952308 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:07.952240    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbq8n\" (UniqueName: \"kubernetes.io/projected/9f4bebec-f960-4df0-8ccf-99b41efe2989-kube-api-access-nbq8n\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:02:07.952308 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:07.952254    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f4bebec-f960-4df0-8ccf-99b41efe2989-tls-certs\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:02:07.952308 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:07.952265    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f4bebec-f960-4df0-8ccf-99b41efe2989-tmp-dir\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:02:07.952308 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:07.952277    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f4bebec-f960-4df0-8ccf-99b41efe2989-home\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:02:07.952308 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:07.952288    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f4bebec-f960-4df0-8ccf-99b41efe2989-model-cache\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:02:08.463338 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:08.463304    2580 generic.go:356] "Generic (PLEG): container finished" podID="9f4bebec-f960-4df0-8ccf-99b41efe2989" containerID="563a2f02b3045fb803c131dc6062536ad73d4f1bc7d4b0163c167f01af974525" exitCode=0
Jul 15 17:02:08.463747 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:08.463367    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"
Jul 15 17:02:08.463747 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:08.463391    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8" event={"ID":"9f4bebec-f960-4df0-8ccf-99b41efe2989","Type":"ContainerDied","Data":"563a2f02b3045fb803c131dc6062536ad73d4f1bc7d4b0163c167f01af974525"}
Jul 15 17:02:08.463747 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:08.463438    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8" event={"ID":"9f4bebec-f960-4df0-8ccf-99b41efe2989","Type":"ContainerDied","Data":"f713680e9c2ca546d05fa771a1eb2852ecfcadbb4cb87e79525a979cebc20f21"}
Jul 15 17:02:08.463747 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:08.463459    2580 scope.go:117] "RemoveContainer" containerID="563a2f02b3045fb803c131dc6062536ad73d4f1bc7d4b0163c167f01af974525"
Jul 15 17:02:08.475335 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:08.475313    2580 scope.go:117] "RemoveContainer" containerID="563a2f02b3045fb803c131dc6062536ad73d4f1bc7d4b0163c167f01af974525"
Jul 15 17:02:08.475711 ip-10-0-134-132 kubenswrapper[2580]: E0715 17:02:08.475688    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"563a2f02b3045fb803c131dc6062536ad73d4f1bc7d4b0163c167f01af974525\": container with ID starting with 563a2f02b3045fb803c131dc6062536ad73d4f1bc7d4b0163c167f01af974525 not found: ID does not exist" containerID="563a2f02b3045fb803c131dc6062536ad73d4f1bc7d4b0163c167f01af974525"
Jul 15 17:02:08.475798 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:08.475723    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"563a2f02b3045fb803c131dc6062536ad73d4f1bc7d4b0163c167f01af974525"} err="failed to get container status \"563a2f02b3045fb803c131dc6062536ad73d4f1bc7d4b0163c167f01af974525\": rpc error: code = NotFound desc = could not find container \"563a2f02b3045fb803c131dc6062536ad73d4f1bc7d4b0163c167f01af974525\": container with ID starting with 563a2f02b3045fb803c131dc6062536ad73d4f1bc7d4b0163c167f01af974525 not found: ID does not exist"
Jul 15 17:02:08.487086 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:08.487064    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"]
Jul 15 17:02:08.490593 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:08.490573    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-8269b5/sub-test-model-8269b5-kserve-55cb6995f5-8p2v8"]
Jul 15 17:02:08.878766 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:08.878706    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f4bebec-f960-4df0-8ccf-99b41efe2989" path="/var/lib/kubelet/pods/9f4bebec-f960-4df0-8ccf-99b41efe2989/volumes"
Jul 15 17:02:09.247382 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:09.247280    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:02:09.247382 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:09.247337    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:02:09.259784 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:09.259746    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:02:09.482313 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:09.482281    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:02:13.298473 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:13.298437    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 17:02:21.638344 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:21.638309    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:02:46.620938 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:46.620896    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:02:49.698554 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:02:49.698517    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 17:03:06.089460 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:06.089379    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 17:03:17.723383 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:17.723339    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:03:17.824991 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:17.824957    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:03:25.888658 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:25.888628    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 17:03:27.555965 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:27.555923    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:03:27.653919 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:27.653883    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:03:29.624918 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:29.624882    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"]
Jul 15 17:03:29.648873 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:29.648835    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:03:30.633866 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:30.633828    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:03:31.661216 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:31.661180    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7410-139944546630576"]
Jul 15 17:03:31.661597 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:31.661523    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f4bebec-f960-4df0-8ccf-99b41efe2989" containerName="main"
Jul 15 17:03:31.661597 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:31.661533    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f4bebec-f960-4df0-8ccf-99b41efe2989" containerName="main"
Jul 15 17:03:31.661597 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:31.661588    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f4bebec-f960-4df0-8ccf-99b41efe2989" containerName="main"
Jul 15 17:03:31.664450 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:31.664434    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-139944546630576"
Jul 15 17:03:31.668008 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:31.667990    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-cmnsb\""
Jul 15 17:03:31.672687 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:31.672667    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7410-139944546630576"]
Jul 15 17:03:31.779452 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:31.779415    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-662c6\" (UniqueName: \"kubernetes.io/projected/4bd5bf02-6af0-4a54-8502-ea13b7d3db2b-kube-api-access-662c6\") pod \"test-curl-7410-139944546630576\" (UID: \"4bd5bf02-6af0-4a54-8502-ea13b7d3db2b\") " pod="opendatahub/test-curl-7410-139944546630576"
Jul 15 17:03:31.880306 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:31.880273    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-662c6\" (UniqueName: \"kubernetes.io/projected/4bd5bf02-6af0-4a54-8502-ea13b7d3db2b-kube-api-access-662c6\") pod \"test-curl-7410-139944546630576\" (UID: \"4bd5bf02-6af0-4a54-8502-ea13b7d3db2b\") " pod="opendatahub/test-curl-7410-139944546630576"
Jul 15 17:03:31.889912 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:31.889888    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-662c6\" (UniqueName: \"kubernetes.io/projected/4bd5bf02-6af0-4a54-8502-ea13b7d3db2b-kube-api-access-662c6\") pod \"test-curl-7410-139944546630576\" (UID: \"4bd5bf02-6af0-4a54-8502-ea13b7d3db2b\") " pod="opendatahub/test-curl-7410-139944546630576"
Jul 15 17:03:31.974495 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:31.974414    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-139944546630576"
Jul 15 17:03:32.104115 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:32.104089    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7410-139944546630576"]
Jul 15 17:03:32.105818 ip-10-0-134-132 kubenswrapper[2580]: W0715 17:03:32.105781    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4bd5bf02_6af0_4a54_8502_ea13b7d3db2b.slice/crio-b27fa4cd7ea24058265349764470fc5b9043762de0b08dedbfe7521f4c333625 WatchSource:0}: Error finding container b27fa4cd7ea24058265349764470fc5b9043762de0b08dedbfe7521f4c333625: Status 404 returned error can't find the container with id b27fa4cd7ea24058265349764470fc5b9043762de0b08dedbfe7521f4c333625
Jul 15 17:03:32.107152 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:32.107132    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 17:03:32.910391 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:32.910360    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7410-139944546630576" event={"ID":"4bd5bf02-6af0-4a54-8502-ea13b7d3db2b","Type":"ContainerStarted","Data":"b27fa4cd7ea24058265349764470fc5b9043762de0b08dedbfe7521f4c333625"}
Jul 15 17:03:33.917789 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:33.917738    2580 generic.go:356] "Generic (PLEG): container finished" podID="4bd5bf02-6af0-4a54-8502-ea13b7d3db2b" containerID="60d44dd37b521e113d2362f89e8dbd625d9089dbdb14a24cb674f282b44bb512" exitCode=0
Jul 15 17:03:33.918174 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:33.917799    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7410-139944546630576" event={"ID":"4bd5bf02-6af0-4a54-8502-ea13b7d3db2b","Type":"ContainerDied","Data":"60d44dd37b521e113d2362f89e8dbd625d9089dbdb14a24cb674f282b44bb512"}
Jul 15 17:03:35.051667 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:35.051642    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-139944546630576"
Jul 15 17:03:35.072257 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:35.072227    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7410-139944546630576_4bd5bf02-6af0-4a54-8502-ea13b7d3db2b/test-curl-7410-139944546630576/0.log"
Jul 15 17:03:35.098324 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:35.098274    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7410-139944546630576"]
Jul 15 17:03:35.100512 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:35.100491    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7410-139944546630576"]
Jul 15 17:03:35.112263 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:35.112242    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-662c6\" (UniqueName: \"kubernetes.io/projected/4bd5bf02-6af0-4a54-8502-ea13b7d3db2b-kube-api-access-662c6\") pod \"4bd5bf02-6af0-4a54-8502-ea13b7d3db2b\" (UID: \"4bd5bf02-6af0-4a54-8502-ea13b7d3db2b\") "
Jul 15 17:03:35.114370 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:35.114341    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bd5bf02-6af0-4a54-8502-ea13b7d3db2b-kube-api-access-662c6" (OuterVolumeSpecName: "kube-api-access-662c6") pod "4bd5bf02-6af0-4a54-8502-ea13b7d3db2b" (UID: "4bd5bf02-6af0-4a54-8502-ea13b7d3db2b"). InnerVolumeSpecName "kube-api-access-662c6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:03:35.213357 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:35.213252    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-662c6\" (UniqueName: \"kubernetes.io/projected/4bd5bf02-6af0-4a54-8502-ea13b7d3db2b-kube-api-access-662c6\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:03:35.229971 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:35.229936    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7410-139944546677296"]
Jul 15 17:03:35.230304 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:35.230291    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bd5bf02-6af0-4a54-8502-ea13b7d3db2b" containerName="test-curl-7410-139944546630576"
Jul 15 17:03:35.230347 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:35.230306    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bd5bf02-6af0-4a54-8502-ea13b7d3db2b" containerName="test-curl-7410-139944546630576"
Jul 15 17:03:35.230382 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:35.230371    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bd5bf02-6af0-4a54-8502-ea13b7d3db2b" containerName="test-curl-7410-139944546630576"
Jul 15 17:03:35.233346 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:35.233327    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-139944546677296"
Jul 15 17:03:35.239576 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:35.239552    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7410-139944546677296"]
Jul 15 17:03:35.314591 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:35.314550    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dk2d\" (UniqueName: \"kubernetes.io/projected/05d9c86d-2af7-4551-a533-c4ef607b05b9-kube-api-access-5dk2d\") pod \"test-curl-7410-139944546677296\" (UID: \"05d9c86d-2af7-4551-a533-c4ef607b05b9\") " pod="opendatahub/test-curl-7410-139944546677296"
Jul 15 17:03:35.415832 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:35.415787    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dk2d\" (UniqueName: \"kubernetes.io/projected/05d9c86d-2af7-4551-a533-c4ef607b05b9-kube-api-access-5dk2d\") pod \"test-curl-7410-139944546677296\" (UID: \"05d9c86d-2af7-4551-a533-c4ef607b05b9\") " pod="opendatahub/test-curl-7410-139944546677296"
Jul 15 17:03:35.424907 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:35.424872    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dk2d\" (UniqueName: \"kubernetes.io/projected/05d9c86d-2af7-4551-a533-c4ef607b05b9-kube-api-access-5dk2d\") pod \"test-curl-7410-139944546677296\" (UID: \"05d9c86d-2af7-4551-a533-c4ef607b05b9\") " pod="opendatahub/test-curl-7410-139944546677296"
Jul 15 17:03:35.543703 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:35.543623    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-139944546677296"
Jul 15 17:03:35.683712 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:35.683677    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7410-139944546677296"]
Jul 15 17:03:35.683872 ip-10-0-134-132 kubenswrapper[2580]: W0715 17:03:35.683832    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05d9c86d_2af7_4551_a533_c4ef607b05b9.slice/crio-94bbfbe44a7c701b99594b8a195dca11374037896c7cb3fe4e6b5fd77cdae24b WatchSource:0}: Error finding container 94bbfbe44a7c701b99594b8a195dca11374037896c7cb3fe4e6b5fd77cdae24b: Status 404 returned error can't find the container with id 94bbfbe44a7c701b99594b8a195dca11374037896c7cb3fe4e6b5fd77cdae24b
Jul 15 17:03:35.929918 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:35.929881    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b27fa4cd7ea24058265349764470fc5b9043762de0b08dedbfe7521f4c333625"
Jul 15 17:03:35.929918 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:35.929882    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-139944546630576"
Jul 15 17:03:35.931203 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:35.931171    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7410-139944546677296" event={"ID":"05d9c86d-2af7-4551-a533-c4ef607b05b9","Type":"ContainerStarted","Data":"94bbfbe44a7c701b99594b8a195dca11374037896c7cb3fe4e6b5fd77cdae24b"}
Jul 15 17:03:35.940730 ip-10-0-134-132 kubenswrapper[2580]: E0715 17:03:35.940701    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7410-139944546630576\" is forbidden: User \"system:node:ip-10-0-134-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-132.ec2.internal' and this object" podUID="4bd5bf02-6af0-4a54-8502-ea13b7d3db2b" pod="opendatahub/test-curl-7410-139944546630576"
Jul 15 17:03:36.879048 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:36.879012    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bd5bf02-6af0-4a54-8502-ea13b7d3db2b" path="/var/lib/kubelet/pods/4bd5bf02-6af0-4a54-8502-ea13b7d3db2b/volumes"
Jul 15 17:03:36.938298 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:36.938261    2580 generic.go:356] "Generic (PLEG): container finished" podID="05d9c86d-2af7-4551-a533-c4ef607b05b9" containerID="06c9f1501f45471b539c5eccd75ab5fb9bed8026b0b4c04d73f4abe6506f7cb8" exitCode=0
Jul 15 17:03:36.938479 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:36.938343    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7410-139944546677296" event={"ID":"05d9c86d-2af7-4551-a533-c4ef607b05b9","Type":"ContainerDied","Data":"06c9f1501f45471b539c5eccd75ab5fb9bed8026b0b4c04d73f4abe6506f7cb8"}
Jul 15 17:03:38.078579 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:38.078550    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-139944546677296"
Jul 15 17:03:38.103741 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:38.103711    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7410-139944546677296_05d9c86d-2af7-4551-a533-c4ef607b05b9/test-curl-7410-139944546677296/0.log"
Jul 15 17:03:38.132300 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:38.132267    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7410-139944546677296"]
Jul 15 17:03:38.134446 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:38.134421    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7410-139944546677296"]
Jul 15 17:03:38.140344 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:38.140322    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dk2d\" (UniqueName: \"kubernetes.io/projected/05d9c86d-2af7-4551-a533-c4ef607b05b9-kube-api-access-5dk2d\") pod \"05d9c86d-2af7-4551-a533-c4ef607b05b9\" (UID: \"05d9c86d-2af7-4551-a533-c4ef607b05b9\") "
Jul 15 17:03:38.142435 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:38.142407    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05d9c86d-2af7-4551-a533-c4ef607b05b9-kube-api-access-5dk2d" (OuterVolumeSpecName: "kube-api-access-5dk2d") pod "05d9c86d-2af7-4551-a533-c4ef607b05b9" (UID: "05d9c86d-2af7-4551-a533-c4ef607b05b9"). InnerVolumeSpecName "kube-api-access-5dk2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:03:38.241229 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:38.241140    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dk2d\" (UniqueName: \"kubernetes.io/projected/05d9c86d-2af7-4551-a533-c4ef607b05b9-kube-api-access-5dk2d\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:03:38.301273 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:38.301238    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7410-139944546656304"]
Jul 15 17:03:38.301610 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:38.301596    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05d9c86d-2af7-4551-a533-c4ef607b05b9" containerName="test-curl-7410-139944546677296"
Jul 15 17:03:38.301653 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:38.301611    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="05d9c86d-2af7-4551-a533-c4ef607b05b9" containerName="test-curl-7410-139944546677296"
Jul 15 17:03:38.301694 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:38.301673    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="05d9c86d-2af7-4551-a533-c4ef607b05b9" containerName="test-curl-7410-139944546677296"
Jul 15 17:03:38.304727 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:38.304711    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-139944546656304"
Jul 15 17:03:38.312622 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:38.312595    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7410-139944546656304"]
Jul 15 17:03:38.443018 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:38.442983    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzctt\" (UniqueName: \"kubernetes.io/projected/48b09377-8388-4cd8-9ffd-8f51530d40e2-kube-api-access-qzctt\") pod \"test-curl-7410-139944546656304\" (UID: \"48b09377-8388-4cd8-9ffd-8f51530d40e2\") " pod="opendatahub/test-curl-7410-139944546656304"
Jul 15 17:03:38.544530 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:38.544433    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzctt\" (UniqueName: \"kubernetes.io/projected/48b09377-8388-4cd8-9ffd-8f51530d40e2-kube-api-access-qzctt\") pod \"test-curl-7410-139944546656304\" (UID: \"48b09377-8388-4cd8-9ffd-8f51530d40e2\") " pod="opendatahub/test-curl-7410-139944546656304"
Jul 15 17:03:38.558373 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:38.558340    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzctt\" (UniqueName: \"kubernetes.io/projected/48b09377-8388-4cd8-9ffd-8f51530d40e2-kube-api-access-qzctt\") pod \"test-curl-7410-139944546656304\" (UID: \"48b09377-8388-4cd8-9ffd-8f51530d40e2\") " pod="opendatahub/test-curl-7410-139944546656304"
Jul 15 17:03:38.614827 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:38.614790    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-139944546656304"
Jul 15 17:03:38.879439 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:38.879406    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05d9c86d-2af7-4551-a533-c4ef607b05b9" path="/var/lib/kubelet/pods/05d9c86d-2af7-4551-a533-c4ef607b05b9/volumes"
Jul 15 17:03:38.950169 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:38.949800    2580 scope.go:117] "RemoveContainer" containerID="06c9f1501f45471b539c5eccd75ab5fb9bed8026b0b4c04d73f4abe6506f7cb8"
Jul 15 17:03:38.950169 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:38.949975    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-139944546677296"
Jul 15 17:03:38.953698 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:38.953678    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7410-139944546656304"]
Jul 15 17:03:38.955396 ip-10-0-134-132 kubenswrapper[2580]: W0715 17:03:38.955368    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48b09377_8388_4cd8_9ffd_8f51530d40e2.slice/crio-85437f55e1ff3de061a1ed04c2b465eb2ac7a05381856a7d14f252527cac1356 WatchSource:0}: Error finding container 85437f55e1ff3de061a1ed04c2b465eb2ac7a05381856a7d14f252527cac1356: Status 404 returned error can't find the container with id 85437f55e1ff3de061a1ed04c2b465eb2ac7a05381856a7d14f252527cac1356
Jul 15 17:03:39.956300 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:39.956264    2580 generic.go:356] "Generic (PLEG): container finished" podID="48b09377-8388-4cd8-9ffd-8f51530d40e2" containerID="1c05d396ca0901180982e4f68e5fdd9f81795423f06a446bf5993be594508185" exitCode=0
Jul 15 17:03:39.956788 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:39.956351    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7410-139944546656304" event={"ID":"48b09377-8388-4cd8-9ffd-8f51530d40e2","Type":"ContainerDied","Data":"1c05d396ca0901180982e4f68e5fdd9f81795423f06a446bf5993be594508185"}
Jul 15 17:03:39.956788 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:39.956390    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7410-139944546656304" event={"ID":"48b09377-8388-4cd8-9ffd-8f51530d40e2","Type":"ContainerStarted","Data":"85437f55e1ff3de061a1ed04c2b465eb2ac7a05381856a7d14f252527cac1356"}
Jul 15 17:03:41.096503 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:41.096477    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-139944546656304"
Jul 15 17:03:41.122978 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:41.122950    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7410-139944546656304_48b09377-8388-4cd8-9ffd-8f51530d40e2/test-curl-7410-139944546656304/0.log"
Jul 15 17:03:41.159873 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:41.159842    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7410-139944546656304"]
Jul 15 17:03:41.166081 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:41.166057    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7410-139944546656304"]
Jul 15 17:03:41.168976 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:41.168958    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzctt\" (UniqueName: \"kubernetes.io/projected/48b09377-8388-4cd8-9ffd-8f51530d40e2-kube-api-access-qzctt\") pod \"48b09377-8388-4cd8-9ffd-8f51530d40e2\" (UID: \"48b09377-8388-4cd8-9ffd-8f51530d40e2\") "
Jul 15 17:03:41.171028 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:41.171001    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48b09377-8388-4cd8-9ffd-8f51530d40e2-kube-api-access-qzctt" (OuterVolumeSpecName: "kube-api-access-qzctt") pod "48b09377-8388-4cd8-9ffd-8f51530d40e2" (UID: "48b09377-8388-4cd8-9ffd-8f51530d40e2"). InnerVolumeSpecName "kube-api-access-qzctt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:03:41.269612 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:41.269525    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzctt\" (UniqueName: \"kubernetes.io/projected/48b09377-8388-4cd8-9ffd-8f51530d40e2-kube-api-access-qzctt\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:03:41.287748 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:41.287712    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7410-139944546656304"]
Jul 15 17:03:41.288093 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:41.288079    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48b09377-8388-4cd8-9ffd-8f51530d40e2" containerName="test-curl-7410-139944546656304"
Jul 15 17:03:41.288150 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:41.288094    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="48b09377-8388-4cd8-9ffd-8f51530d40e2" containerName="test-curl-7410-139944546656304"
Jul 15 17:03:41.288187 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:41.288165    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="48b09377-8388-4cd8-9ffd-8f51530d40e2" containerName="test-curl-7410-139944546656304"
Jul 15 17:03:41.291063 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:41.291047    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-139944546656304"
Jul 15 17:03:41.299722 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:41.299699    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7410-139944546656304"]
Jul 15 17:03:41.370307 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:41.370270    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfhlk\" (UniqueName: \"kubernetes.io/projected/b5ae10ca-ce0b-4aad-883b-e113d23b8a43-kube-api-access-hfhlk\") pod \"test-curl-7410-139944546656304\" (UID: \"b5ae10ca-ce0b-4aad-883b-e113d23b8a43\") " pod="opendatahub/test-curl-7410-139944546656304"
Jul 15 17:03:41.471360 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:41.471324    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfhlk\" (UniqueName: \"kubernetes.io/projected/b5ae10ca-ce0b-4aad-883b-e113d23b8a43-kube-api-access-hfhlk\") pod \"test-curl-7410-139944546656304\" (UID: \"b5ae10ca-ce0b-4aad-883b-e113d23b8a43\") " pod="opendatahub/test-curl-7410-139944546656304"
Jul 15 17:03:41.486236 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:41.486212    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfhlk\" (UniqueName: \"kubernetes.io/projected/b5ae10ca-ce0b-4aad-883b-e113d23b8a43-kube-api-access-hfhlk\") pod \"test-curl-7410-139944546656304\" (UID: \"b5ae10ca-ce0b-4aad-883b-e113d23b8a43\") " pod="opendatahub/test-curl-7410-139944546656304"
Jul 15 17:03:41.600848 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:41.600817    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-139944546656304"
Jul 15 17:03:41.746125 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:41.746102    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7410-139944546656304"]
Jul 15 17:03:41.748167 ip-10-0-134-132 kubenswrapper[2580]: W0715 17:03:41.748137    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5ae10ca_ce0b_4aad_883b_e113d23b8a43.slice/crio-3fd96a29c8f638541afa7894e99b49523774761abc6bb9360d809bc22fefbc40 WatchSource:0}: Error finding container 3fd96a29c8f638541afa7894e99b49523774761abc6bb9360d809bc22fefbc40: Status 404 returned error can't find the container with id 3fd96a29c8f638541afa7894e99b49523774761abc6bb9360d809bc22fefbc40
Jul 15 17:03:41.968417 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:41.968321    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7410-139944546656304" event={"ID":"b5ae10ca-ce0b-4aad-883b-e113d23b8a43","Type":"ContainerStarted","Data":"3fd96a29c8f638541afa7894e99b49523774761abc6bb9360d809bc22fefbc40"}
Jul 15 17:03:41.969503 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:41.969488    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="85437f55e1ff3de061a1ed04c2b465eb2ac7a05381856a7d14f252527cac1356"
Jul 15 17:03:41.969581 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:41.969513    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-139944546656304"
Jul 15 17:03:41.981706 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:41.981673    2580 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7410-139944546656304" oldPodUID="48b09377-8388-4cd8-9ffd-8f51530d40e2" podUID="b5ae10ca-ce0b-4aad-883b-e113d23b8a43"
Jul 15 17:03:42.296001 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:42.295902    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 17:03:42.879128 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:42.879087    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48b09377-8388-4cd8-9ffd-8f51530d40e2" path="/var/lib/kubelet/pods/48b09377-8388-4cd8-9ffd-8f51530d40e2/volumes"
Jul 15 17:03:42.976248 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:42.976207    2580 generic.go:356] "Generic (PLEG): container finished" podID="b5ae10ca-ce0b-4aad-883b-e113d23b8a43" containerID="ae9a6b324fe4c6e8e2c02674f05f91862dff072a229ae78d929c051ff420ff9c" exitCode=0
Jul 15 17:03:42.976400 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:42.976293    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7410-139944546656304" event={"ID":"b5ae10ca-ce0b-4aad-883b-e113d23b8a43","Type":"ContainerDied","Data":"ae9a6b324fe4c6e8e2c02674f05f91862dff072a229ae78d929c051ff420ff9c"}
Jul 15 17:03:44.116626 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.116597    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-139944546656304"
Jul 15 17:03:44.140907 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.140875    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7410-139944546656304_b5ae10ca-ce0b-4aad-883b-e113d23b8a43/test-curl-7410-139944546656304/0.log"
Jul 15 17:03:44.171004 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.170969    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7410-139944546656304"]
Jul 15 17:03:44.173615 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.173589    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7410-139944546656304"]
Jul 15 17:03:44.196807 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.196752    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfhlk\" (UniqueName: \"kubernetes.io/projected/b5ae10ca-ce0b-4aad-883b-e113d23b8a43-kube-api-access-hfhlk\") pod \"b5ae10ca-ce0b-4aad-883b-e113d23b8a43\" (UID: \"b5ae10ca-ce0b-4aad-883b-e113d23b8a43\") "
Jul 15 17:03:44.198777 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.198740    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5ae10ca-ce0b-4aad-883b-e113d23b8a43-kube-api-access-hfhlk" (OuterVolumeSpecName: "kube-api-access-hfhlk") pod "b5ae10ca-ce0b-4aad-883b-e113d23b8a43" (UID: "b5ae10ca-ce0b-4aad-883b-e113d23b8a43"). InnerVolumeSpecName "kube-api-access-hfhlk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:03:44.297959 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.297873    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfhlk\" (UniqueName: \"kubernetes.io/projected/b5ae10ca-ce0b-4aad-883b-e113d23b8a43-kube-api-access-hfhlk\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:03:44.323473 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.323444    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7410-139944546657312"]
Jul 15 17:03:44.323823 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.323806    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5ae10ca-ce0b-4aad-883b-e113d23b8a43" containerName="test-curl-7410-139944546656304"
Jul 15 17:03:44.323878 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.323825    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5ae10ca-ce0b-4aad-883b-e113d23b8a43" containerName="test-curl-7410-139944546656304"
Jul 15 17:03:44.323927 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.323910    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5ae10ca-ce0b-4aad-883b-e113d23b8a43" containerName="test-curl-7410-139944546656304"
Jul 15 17:03:44.326930 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.326907    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-139944546657312"
Jul 15 17:03:44.340019 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.339990    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7410-139944546657312"]
Jul 15 17:03:44.499850 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.499814    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvds8\" (UniqueName: \"kubernetes.io/projected/c3edbe95-0555-4d41-8a83-a12a4abf7e45-kube-api-access-vvds8\") pod \"test-curl-7410-139944546657312\" (UID: \"c3edbe95-0555-4d41-8a83-a12a4abf7e45\") " pod="opendatahub/test-curl-7410-139944546657312"
Jul 15 17:03:44.600849 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.600813    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvds8\" (UniqueName: \"kubernetes.io/projected/c3edbe95-0555-4d41-8a83-a12a4abf7e45-kube-api-access-vvds8\") pod \"test-curl-7410-139944546657312\" (UID: \"c3edbe95-0555-4d41-8a83-a12a4abf7e45\") " pod="opendatahub/test-curl-7410-139944546657312"
Jul 15 17:03:44.610728 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.610691    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvds8\" (UniqueName: \"kubernetes.io/projected/c3edbe95-0555-4d41-8a83-a12a4abf7e45-kube-api-access-vvds8\") pod \"test-curl-7410-139944546657312\" (UID: \"c3edbe95-0555-4d41-8a83-a12a4abf7e45\") " pod="opendatahub/test-curl-7410-139944546657312"
Jul 15 17:03:44.636053 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.636020    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-139944546657312"
Jul 15 17:03:44.646362 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.646330    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q" podUID="db3880df-d8cc-4759-b567-baf9cf57eab1" containerName="main" containerID="cri-o://ce347657d4e1d7dd240895509c1a9980ce61c0827e054a147396a73c9eb96fd0" gracePeriod=45
Jul 15 17:03:44.771309 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.771284    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7410-139944546657312"]
Jul 15 17:03:44.772920 ip-10-0-134-132 kubenswrapper[2580]: W0715 17:03:44.772890    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3edbe95_0555_4d41_8a83_a12a4abf7e45.slice/crio-c3c507ba1b66d1be81c5eb1f102fba585f6f61319b3055e628c600456af89425 WatchSource:0}: Error finding container c3c507ba1b66d1be81c5eb1f102fba585f6f61319b3055e628c600456af89425: Status 404 returned error can't find the container with id c3c507ba1b66d1be81c5eb1f102fba585f6f61319b3055e628c600456af89425
Jul 15 17:03:44.879491 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.879464    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5ae10ca-ce0b-4aad-883b-e113d23b8a43" path="/var/lib/kubelet/pods/b5ae10ca-ce0b-4aad-883b-e113d23b8a43/volumes"
Jul 15 17:03:44.884852 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.884829    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:03:44.987670 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.987625    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7410-139944546657312" event={"ID":"c3edbe95-0555-4d41-8a83-a12a4abf7e45","Type":"ContainerStarted","Data":"c3c507ba1b66d1be81c5eb1f102fba585f6f61319b3055e628c600456af89425"}
Jul 15 17:03:44.988880 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.988862    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-139944546656304"
Jul 15 17:03:44.989005 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.988887    2580 scope.go:117] "RemoveContainer" containerID="ae9a6b324fe4c6e8e2c02674f05f91862dff072a229ae78d929c051ff420ff9c"
Jul 15 17:03:44.990392 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.990363    2580 generic.go:356] "Generic (PLEG): container finished" podID="db3880df-d8cc-4759-b567-baf9cf57eab1" containerID="ce347657d4e1d7dd240895509c1a9980ce61c0827e054a147396a73c9eb96fd0" exitCode=0
Jul 15 17:03:44.990495 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.990414    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"
Jul 15 17:03:44.990495 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.990431    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q" event={"ID":"db3880df-d8cc-4759-b567-baf9cf57eab1","Type":"ContainerDied","Data":"ce347657d4e1d7dd240895509c1a9980ce61c0827e054a147396a73c9eb96fd0"}
Jul 15 17:03:44.990495 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.990460    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q" event={"ID":"db3880df-d8cc-4759-b567-baf9cf57eab1","Type":"ContainerDied","Data":"239a3a373e9bb04b37cf6a811a3b5a6037cebe29c649fccb72a50c242ccf603a"}
Jul 15 17:03:44.997655 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:44.997636    2580 scope.go:117] "RemoveContainer" containerID="ce347657d4e1d7dd240895509c1a9980ce61c0827e054a147396a73c9eb96fd0"
Jul 15 17:03:45.003924 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:45.003895    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db3880df-d8cc-4759-b567-baf9cf57eab1-model-cache\") pod \"db3880df-d8cc-4759-b567-baf9cf57eab1\" (UID: \"db3880df-d8cc-4759-b567-baf9cf57eab1\") "
Jul 15 17:03:45.004021 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:45.003932    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db3880df-d8cc-4759-b567-baf9cf57eab1-dshm\") pod \"db3880df-d8cc-4759-b567-baf9cf57eab1\" (UID: \"db3880df-d8cc-4759-b567-baf9cf57eab1\") "
Jul 15 17:03:45.004021 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:45.003984    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db3880df-d8cc-4759-b567-baf9cf57eab1-home\") pod \"db3880df-d8cc-4759-b567-baf9cf57eab1\" (UID: \"db3880df-d8cc-4759-b567-baf9cf57eab1\") "
Jul 15 17:03:45.004021 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:45.004006    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db3880df-d8cc-4759-b567-baf9cf57eab1-tls-certs\") pod \"db3880df-d8cc-4759-b567-baf9cf57eab1\" (UID: \"db3880df-d8cc-4759-b567-baf9cf57eab1\") "
Jul 15 17:03:45.004180 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:45.004058    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2gkx\" (UniqueName: \"kubernetes.io/projected/db3880df-d8cc-4759-b567-baf9cf57eab1-kube-api-access-f2gkx\") pod \"db3880df-d8cc-4759-b567-baf9cf57eab1\" (UID: \"db3880df-d8cc-4759-b567-baf9cf57eab1\") "
Jul 15 17:03:45.004180 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:45.004104    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db3880df-d8cc-4759-b567-baf9cf57eab1-tmp-dir\") pod \"db3880df-d8cc-4759-b567-baf9cf57eab1\" (UID: \"db3880df-d8cc-4759-b567-baf9cf57eab1\") "
Jul 15 17:03:45.004180 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:45.004140    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db3880df-d8cc-4759-b567-baf9cf57eab1-model-cache" (OuterVolumeSpecName: "model-cache") pod "db3880df-d8cc-4759-b567-baf9cf57eab1" (UID: "db3880df-d8cc-4759-b567-baf9cf57eab1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:03:45.004328 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:45.004231    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db3880df-d8cc-4759-b567-baf9cf57eab1-home" (OuterVolumeSpecName: "home") pod "db3880df-d8cc-4759-b567-baf9cf57eab1" (UID: "db3880df-d8cc-4759-b567-baf9cf57eab1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:03:45.004446 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:45.004423    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db3880df-d8cc-4759-b567-baf9cf57eab1-model-cache\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:03:45.004566 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:45.004458    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db3880df-d8cc-4759-b567-baf9cf57eab1-home\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:03:45.004566 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:45.004428    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db3880df-d8cc-4759-b567-baf9cf57eab1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "db3880df-d8cc-4759-b567-baf9cf57eab1" (UID: "db3880df-d8cc-4759-b567-baf9cf57eab1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:03:45.006122 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:45.006091    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db3880df-d8cc-4759-b567-baf9cf57eab1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "db3880df-d8cc-4759-b567-baf9cf57eab1" (UID: "db3880df-d8cc-4759-b567-baf9cf57eab1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:03:45.006208 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:45.006177    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db3880df-d8cc-4759-b567-baf9cf57eab1-kube-api-access-f2gkx" (OuterVolumeSpecName: "kube-api-access-f2gkx") pod "db3880df-d8cc-4759-b567-baf9cf57eab1" (UID: "db3880df-d8cc-4759-b567-baf9cf57eab1"). InnerVolumeSpecName "kube-api-access-f2gkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:03:45.006360 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:45.006342    2580 scope.go:117] "RemoveContainer" containerID="ce347657d4e1d7dd240895509c1a9980ce61c0827e054a147396a73c9eb96fd0"
Jul 15 17:03:45.006463 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:45.006415    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db3880df-d8cc-4759-b567-baf9cf57eab1-dshm" (OuterVolumeSpecName: "dshm") pod "db3880df-d8cc-4759-b567-baf9cf57eab1" (UID: "db3880df-d8cc-4759-b567-baf9cf57eab1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 17:03:45.006641 ip-10-0-134-132 kubenswrapper[2580]: E0715 17:03:45.006618    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce347657d4e1d7dd240895509c1a9980ce61c0827e054a147396a73c9eb96fd0\": container with ID starting with ce347657d4e1d7dd240895509c1a9980ce61c0827e054a147396a73c9eb96fd0 not found: ID does not exist" containerID="ce347657d4e1d7dd240895509c1a9980ce61c0827e054a147396a73c9eb96fd0"
Jul 15 17:03:45.006697 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:45.006653    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce347657d4e1d7dd240895509c1a9980ce61c0827e054a147396a73c9eb96fd0"} err="failed to get container status \"ce347657d4e1d7dd240895509c1a9980ce61c0827e054a147396a73c9eb96fd0\": rpc error: code = NotFound desc = could not find container \"ce347657d4e1d7dd240895509c1a9980ce61c0827e054a147396a73c9eb96fd0\": container with ID starting with ce347657d4e1d7dd240895509c1a9980ce61c0827e054a147396a73c9eb96fd0 not found: ID does not exist"
Jul 15 17:03:45.105097 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:45.105063    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db3880df-d8cc-4759-b567-baf9cf57eab1-tls-certs\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:03:45.105221 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:45.105103    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2gkx\" (UniqueName: \"kubernetes.io/projected/db3880df-d8cc-4759-b567-baf9cf57eab1-kube-api-access-f2gkx\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:03:45.105221 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:45.105146    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db3880df-d8cc-4759-b567-baf9cf57eab1-tmp-dir\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:03:45.105221 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:45.105163    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db3880df-d8cc-4759-b567-baf9cf57eab1-dshm\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:03:45.314811 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:45.314786    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"]
Jul 15 17:03:45.317688 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:45.317663    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-8269b5/rate-test-model-8269b5-kserve-6fdf68f79-fbk7q"]
Jul 15 17:03:45.998189 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:45.998152    2580 generic.go:356] "Generic (PLEG): container finished" podID="c3edbe95-0555-4d41-8a83-a12a4abf7e45" containerID="02498e9cb6b31e7127c27bd6698770139d12d7d1e5239f66053bf1770aa022b6" exitCode=0
Jul 15 17:03:45.998381 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:45.998237    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7410-139944546657312" event={"ID":"c3edbe95-0555-4d41-8a83-a12a4abf7e45","Type":"ContainerDied","Data":"02498e9cb6b31e7127c27bd6698770139d12d7d1e5239f66053bf1770aa022b6"}
Jul 15 17:03:46.879579 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:46.879542    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db3880df-d8cc-4759-b567-baf9cf57eab1" path="/var/lib/kubelet/pods/db3880df-d8cc-4759-b567-baf9cf57eab1/volumes"
Jul 15 17:03:47.142722 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:47.142662    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-139944546657312"
Jul 15 17:03:47.176362 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:47.176322    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7410-139944546657312_c3edbe95-0555-4d41-8a83-a12a4abf7e45/test-curl-7410-139944546657312/0.log"
Jul 15 17:03:47.209345 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:47.209313    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7410-139944546657312"]
Jul 15 17:03:47.214202 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:47.214175    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7410-139944546657312"]
Jul 15 17:03:47.325415 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:47.325382    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvds8\" (UniqueName: \"kubernetes.io/projected/c3edbe95-0555-4d41-8a83-a12a4abf7e45-kube-api-access-vvds8\") pod \"c3edbe95-0555-4d41-8a83-a12a4abf7e45\" (UID: \"c3edbe95-0555-4d41-8a83-a12a4abf7e45\") "
Jul 15 17:03:47.327533 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:47.327506    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3edbe95-0555-4d41-8a83-a12a4abf7e45-kube-api-access-vvds8" (OuterVolumeSpecName: "kube-api-access-vvds8") pod "c3edbe95-0555-4d41-8a83-a12a4abf7e45" (UID: "c3edbe95-0555-4d41-8a83-a12a4abf7e45"). InnerVolumeSpecName "kube-api-access-vvds8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:03:47.426861 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:47.426774    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvds8\" (UniqueName: \"kubernetes.io/projected/c3edbe95-0555-4d41-8a83-a12a4abf7e45-kube-api-access-vvds8\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:03:48.010605 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:48.010575    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c3c507ba1b66d1be81c5eb1f102fba585f6f61319b3055e628c600456af89425"
Jul 15 17:03:48.011117 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:48.010612    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-139944546657312"
Jul 15 17:03:48.020812 ip-10-0-134-132 kubenswrapper[2580]: E0715 17:03:48.020785    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7410-139944546657312\" is forbidden: User \"system:node:ip-10-0-134-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-132.ec2.internal' and this object" podUID="c3edbe95-0555-4d41-8a83-a12a4abf7e45" pod="opendatahub/test-curl-7410-139944546657312"
Jul 15 17:03:48.878979 ip-10-0-134-132 kubenswrapper[2580]: E0715 17:03:48.878942    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7410-139944546657312\" is forbidden: User \"system:node:ip-10-0-134-132.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-132.ec2.internal' and this object" podUID="c3edbe95-0555-4d41-8a83-a12a4abf7e45" pod="opendatahub/test-curl-7410-139944546657312"
Jul 15 17:03:48.882133 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:48.882096    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3edbe95-0555-4d41-8a83-a12a4abf7e45" path="/var/lib/kubelet/pods/c3edbe95-0555-4d41-8a83-a12a4abf7e45/volumes"
Jul 15 17:03:50.854553 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:50.854514    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8269b5-mgjg5"]
Jul 15 17:03:50.855080 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:50.855059    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db3880df-d8cc-4759-b567-baf9cf57eab1" containerName="main"
Jul 15 17:03:50.855165 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:50.855083    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="db3880df-d8cc-4759-b567-baf9cf57eab1" containerName="main"
Jul 15 17:03:50.855165 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:50.855101    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3edbe95-0555-4d41-8a83-a12a4abf7e45" containerName="test-curl-7410-139944546657312"
Jul 15 17:03:50.855165 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:50.855110    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3edbe95-0555-4d41-8a83-a12a4abf7e45" containerName="test-curl-7410-139944546657312"
Jul 15 17:03:50.855315 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:50.855240    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="db3880df-d8cc-4759-b567-baf9cf57eab1" containerName="main"
Jul 15 17:03:50.855315 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:50.855252    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3edbe95-0555-4d41-8a83-a12a4abf7e45" containerName="test-curl-7410-139944546657312"
Jul 15 17:03:50.858389 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:50.858368    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8269b5-mgjg5"
Jul 15 17:03:50.861050 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:50.861026    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lsp65\""
Jul 15 17:03:50.866650 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:50.866629    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8269b5-mgjg5"]
Jul 15 17:03:50.957427 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:50.957393    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ca7c91e-c6c1-4c33-b7e3-37c2db6ce496-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-8269b5-mgjg5\" (UID: \"3ca7c91e-c6c1-4c33-b7e3-37c2db6ce496\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8269b5-mgjg5"
Jul 15 17:03:51.058513 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:51.058477    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ca7c91e-c6c1-4c33-b7e3-37c2db6ce496-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-8269b5-mgjg5\" (UID: \"3ca7c91e-c6c1-4c33-b7e3-37c2db6ce496\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8269b5-mgjg5"
Jul 15 17:03:51.059164 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:51.059138    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ca7c91e-c6c1-4c33-b7e3-37c2db6ce496-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-8269b5-mgjg5\" (UID: \"3ca7c91e-c6c1-4c33-b7e3-37c2db6ce496\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8269b5-mgjg5"
Jul 15 17:03:51.169697 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:51.169618    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8269b5-mgjg5"
Jul 15 17:03:51.307566 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:51.307541    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8269b5-mgjg5"]
Jul 15 17:03:51.308822 ip-10-0-134-132 kubenswrapper[2580]: W0715 17:03:51.308794    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ca7c91e_c6c1_4c33_b7e3_37c2db6ce496.slice/crio-e782e744a0e4fb72576632e63348c3c06fc93f87b0ee64e3cf539b637df3bf3f WatchSource:0}: Error finding container e782e744a0e4fb72576632e63348c3c06fc93f87b0ee64e3cf539b637df3bf3f: Status 404 returned error can't find the container with id e782e744a0e4fb72576632e63348c3c06fc93f87b0ee64e3cf539b637df3bf3f
Jul 15 17:03:51.404522 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:51.404476    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:03:51.504552 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:51.504475    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:03:52.032384 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:52.032345    2580 generic.go:356] "Generic (PLEG): container finished" podID="3ca7c91e-c6c1-4c33-b7e3-37c2db6ce496" containerID="b4226282a8d48aeea77e77f00e44945077ba3bb91cd2f71b6109bc848fe2862c" exitCode=0
Jul 15 17:03:52.032784 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:52.032436    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8269b5-mgjg5" event={"ID":"3ca7c91e-c6c1-4c33-b7e3-37c2db6ce496","Type":"ContainerDied","Data":"b4226282a8d48aeea77e77f00e44945077ba3bb91cd2f71b6109bc848fe2862c"}
Jul 15 17:03:52.032784 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:52.032505    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8269b5-mgjg5" event={"ID":"3ca7c91e-c6c1-4c33-b7e3-37c2db6ce496","Type":"ContainerStarted","Data":"e782e744a0e4fb72576632e63348c3c06fc93f87b0ee64e3cf539b637df3bf3f"}
Jul 15 17:03:53.166220 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:53.166197    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8269b5-mgjg5"
Jul 15 17:03:53.278839 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:53.278801    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ca7c91e-c6c1-4c33-b7e3-37c2db6ce496-maas-api-service-ca\") pod \"3ca7c91e-c6c1-4c33-b7e3-37c2db6ce496\" (UID: \"3ca7c91e-c6c1-4c33-b7e3-37c2db6ce496\") "
Jul 15 17:03:53.279170 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:53.279148    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ca7c91e-c6c1-4c33-b7e3-37c2db6ce496-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3ca7c91e-c6c1-4c33-b7e3-37c2db6ce496" (UID: "3ca7c91e-c6c1-4c33-b7e3-37c2db6ce496"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:03:53.380195 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:53.380160    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ca7c91e-c6c1-4c33-b7e3-37c2db6ce496-maas-api-service-ca\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:03:54.044234 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:54.044203    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8269b5-mgjg5" event={"ID":"3ca7c91e-c6c1-4c33-b7e3-37c2db6ce496","Type":"ContainerDied","Data":"e782e744a0e4fb72576632e63348c3c06fc93f87b0ee64e3cf539b637df3bf3f"}
Jul 15 17:03:54.044406 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:54.044249    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e782e744a0e4fb72576632e63348c3c06fc93f87b0ee64e3cf539b637df3bf3f"
Jul 15 17:03:54.044406 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:03:54.044211    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8269b5-mgjg5"
Jul 15 17:04:01.324007 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:01.323971    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92"]
Jul 15 17:04:01.324387 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:01.324242    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92" podUID="7a363a7f-1088-4ad2-8a32-3c53a07164aa" containerName="maas-api" containerID="cri-o://1c57cc65cdac604a9c0b61f1c610c09f8b74bc07cc667ad2e2b82043f3808198" gracePeriod=30
Jul 15 17:04:01.356230 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:01.356204    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8269b5-29735580-8dq2g"]
Jul 15 17:04:01.365862 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:01.365837    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8269b5-29735580-8dq2g"]
Jul 15 17:04:01.369828 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:01.369801    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:04:01.586323 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:01.586264    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92"
Jul 15 17:04:01.638645 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:01.638618    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6v6l\" (UniqueName: \"kubernetes.io/projected/7a363a7f-1088-4ad2-8a32-3c53a07164aa-kube-api-access-d6v6l\") pod \"7a363a7f-1088-4ad2-8a32-3c53a07164aa\" (UID: \"7a363a7f-1088-4ad2-8a32-3c53a07164aa\") "
Jul 15 17:04:01.638842 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:01.638661    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a363a7f-1088-4ad2-8a32-3c53a07164aa-openshift-service-ca\") pod \"7a363a7f-1088-4ad2-8a32-3c53a07164aa\" (UID: \"7a363a7f-1088-4ad2-8a32-3c53a07164aa\") "
Jul 15 17:04:01.638842 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:01.638684    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a363a7f-1088-4ad2-8a32-3c53a07164aa-maas-api-tls\") pod \"7a363a7f-1088-4ad2-8a32-3c53a07164aa\" (UID: \"7a363a7f-1088-4ad2-8a32-3c53a07164aa\") "
Jul 15 17:04:01.639097 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:01.639070    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a363a7f-1088-4ad2-8a32-3c53a07164aa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7a363a7f-1088-4ad2-8a32-3c53a07164aa" (UID: "7a363a7f-1088-4ad2-8a32-3c53a07164aa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:04:01.640789 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:01.640746    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a363a7f-1088-4ad2-8a32-3c53a07164aa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7a363a7f-1088-4ad2-8a32-3c53a07164aa" (UID: "7a363a7f-1088-4ad2-8a32-3c53a07164aa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:04:01.640884 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:01.640797    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a363a7f-1088-4ad2-8a32-3c53a07164aa-kube-api-access-d6v6l" (OuterVolumeSpecName: "kube-api-access-d6v6l") pod "7a363a7f-1088-4ad2-8a32-3c53a07164aa" (UID: "7a363a7f-1088-4ad2-8a32-3c53a07164aa"). InnerVolumeSpecName "kube-api-access-d6v6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:04:01.739595 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:01.739561    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6v6l\" (UniqueName: \"kubernetes.io/projected/7a363a7f-1088-4ad2-8a32-3c53a07164aa-kube-api-access-d6v6l\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:04:01.739595 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:01.739588    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a363a7f-1088-4ad2-8a32-3c53a07164aa-openshift-service-ca\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:04:01.739595 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:01.739598    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a363a7f-1088-4ad2-8a32-3c53a07164aa-maas-api-tls\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:04:02.091059 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:02.091019    2580 generic.go:356] "Generic (PLEG): container finished" podID="7a363a7f-1088-4ad2-8a32-3c53a07164aa" containerID="1c57cc65cdac604a9c0b61f1c610c09f8b74bc07cc667ad2e2b82043f3808198" exitCode=0
Jul 15 17:04:02.091059 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:02.091061    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92"
Jul 15 17:04:02.091270 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:02.091085    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92" event={"ID":"7a363a7f-1088-4ad2-8a32-3c53a07164aa","Type":"ContainerDied","Data":"1c57cc65cdac604a9c0b61f1c610c09f8b74bc07cc667ad2e2b82043f3808198"}
Jul 15 17:04:02.091270 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:02.091136    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92" event={"ID":"7a363a7f-1088-4ad2-8a32-3c53a07164aa","Type":"ContainerDied","Data":"5bb2f24abb03f312b6bb311ff89d4c3516e54b99432be6969318ff800994179a"}
Jul 15 17:04:02.091270 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:02.091161    2580 scope.go:117] "RemoveContainer" containerID="1c57cc65cdac604a9c0b61f1c610c09f8b74bc07cc667ad2e2b82043f3808198"
Jul 15 17:04:02.100935 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:02.100915    2580 scope.go:117] "RemoveContainer" containerID="1c57cc65cdac604a9c0b61f1c610c09f8b74bc07cc667ad2e2b82043f3808198"
Jul 15 17:04:02.101174 ip-10-0-134-132 kubenswrapper[2580]: E0715 17:04:02.101158    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c57cc65cdac604a9c0b61f1c610c09f8b74bc07cc667ad2e2b82043f3808198\": container with ID starting with 1c57cc65cdac604a9c0b61f1c610c09f8b74bc07cc667ad2e2b82043f3808198 not found: ID does not exist" containerID="1c57cc65cdac604a9c0b61f1c610c09f8b74bc07cc667ad2e2b82043f3808198"
Jul 15 17:04:02.101235 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:02.101182    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c57cc65cdac604a9c0b61f1c610c09f8b74bc07cc667ad2e2b82043f3808198"} err="failed to get container status \"1c57cc65cdac604a9c0b61f1c610c09f8b74bc07cc667ad2e2b82043f3808198\": rpc error: code = NotFound desc = could not find container \"1c57cc65cdac604a9c0b61f1c610c09f8b74bc07cc667ad2e2b82043f3808198\": container with ID starting with 1c57cc65cdac604a9c0b61f1c610c09f8b74bc07cc667ad2e2b82043f3808198 not found: ID does not exist"
Jul 15 17:04:02.115494 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:02.115464    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92"]
Jul 15 17:04:02.117629 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:02.117602    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8269b5-576d5b5b7f-2lb92"]
Jul 15 17:04:02.878793 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:02.878736    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="793c123e-c10a-4dee-8bd5-edd191b14007" path="/var/lib/kubelet/pods/793c123e-c10a-4dee-8bd5-edd191b14007/volumes"
Jul 15 17:04:02.879279 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:02.879259    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a363a7f-1088-4ad2-8a32-3c53a07164aa" path="/var/lib/kubelet/pods/7a363a7f-1088-4ad2-8a32-3c53a07164aa/volumes"
Jul 15 17:04:05.296263 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:05.296227    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 17:04:05.969640 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:05.969598    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:04:12.283845 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:12.283802    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:04:12.385131 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:12.385097    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:04:16.790948 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:16.790908    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:04:16.997874 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:16.997804    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:04:17.148934 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:17.148899    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-021cd5-x8m44"]
Jul 15 17:04:17.149258 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:17.149245    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ca7c91e-c6c1-4c33-b7e3-37c2db6ce496" containerName="revoke-keys"
Jul 15 17:04:17.149305 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:17.149259    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ca7c91e-c6c1-4c33-b7e3-37c2db6ce496" containerName="revoke-keys"
Jul 15 17:04:17.149305 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:17.149278    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a363a7f-1088-4ad2-8a32-3c53a07164aa" containerName="maas-api"
Jul 15 17:04:17.149305 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:17.149284    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a363a7f-1088-4ad2-8a32-3c53a07164aa" containerName="maas-api"
Jul 15 17:04:17.149401 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:17.149337    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a363a7f-1088-4ad2-8a32-3c53a07164aa" containerName="maas-api"
Jul 15 17:04:17.149401 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:17.149345    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ca7c91e-c6c1-4c33-b7e3-37c2db6ce496" containerName="revoke-keys"
Jul 15 17:04:17.154058 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:17.154037    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-021cd5-x8m44"
Jul 15 17:04:17.160095 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:17.160068    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lsp65\""
Jul 15 17:04:17.172961 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:17.172933    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-021cd5-x8m44"]
Jul 15 17:04:17.273980 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:17.273939    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4c03a4c-ed0f-44d3-86bf-905861139942-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-021cd5-x8m44\" (UID: \"a4c03a4c-ed0f-44d3-86bf-905861139942\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-021cd5-x8m44"
Jul 15 17:04:17.375115 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:17.375075    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4c03a4c-ed0f-44d3-86bf-905861139942-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-021cd5-x8m44\" (UID: \"a4c03a4c-ed0f-44d3-86bf-905861139942\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-021cd5-x8m44"
Jul 15 17:04:17.375710 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:17.375691    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4c03a4c-ed0f-44d3-86bf-905861139942-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-021cd5-x8m44\" (UID: \"a4c03a4c-ed0f-44d3-86bf-905861139942\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-021cd5-x8m44"
Jul 15 17:04:17.463919 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:17.463825    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-021cd5-x8m44"
Jul 15 17:04:17.605226 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:17.605201    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-021cd5-x8m44"]
Jul 15 17:04:17.607887 ip-10-0-134-132 kubenswrapper[2580]: W0715 17:04:17.607854    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4c03a4c_ed0f_44d3_86bf_905861139942.slice/crio-6d165e72ebf926ce33b99c43b48b2ee2e16788adfe0eba59ab1ee4cf7942fec5 WatchSource:0}: Error finding container 6d165e72ebf926ce33b99c43b48b2ee2e16788adfe0eba59ab1ee4cf7942fec5: Status 404 returned error can't find the container with id 6d165e72ebf926ce33b99c43b48b2ee2e16788adfe0eba59ab1ee4cf7942fec5
Jul 15 17:04:17.958175 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:17.958136    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:04:18.121280 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:18.121197    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:04:18.186080 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:18.186043    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-021cd5-x8m44" event={"ID":"a4c03a4c-ed0f-44d3-86bf-905861139942","Type":"ContainerStarted","Data":"8b08886af4e703591c373fac26c8a7604ff202ab80560707611d907e65c36158"}
Jul 15 17:04:18.186265 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:18.186095    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-021cd5-x8m44" event={"ID":"a4c03a4c-ed0f-44d3-86bf-905861139942","Type":"ContainerStarted","Data":"6d165e72ebf926ce33b99c43b48b2ee2e16788adfe0eba59ab1ee4cf7942fec5"}
Jul 15 17:04:18.220604 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:18.220551    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-021cd5-x8m44" podStartSLOduration=1.220537007 podStartE2EDuration="1.220537007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:04:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:04:18.218222661 +0000 UTC m=+2859.865263209" watchObservedRunningTime="2026-07-15 17:04:18.220537007 +0000 UTC m=+2859.867577542"
Jul 15 17:04:19.193999 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:19.193958    2580 generic.go:356] "Generic (PLEG): container finished" podID="a4c03a4c-ed0f-44d3-86bf-905861139942" containerID="8b08886af4e703591c373fac26c8a7604ff202ab80560707611d907e65c36158" exitCode=0
Jul 15 17:04:19.194372 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:19.194042    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-021cd5-x8m44" event={"ID":"a4c03a4c-ed0f-44d3-86bf-905861139942","Type":"ContainerDied","Data":"8b08886af4e703591c373fac26c8a7604ff202ab80560707611d907e65c36158"}
Jul 15 17:04:20.326747 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:20.326721    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-021cd5-x8m44"
Jul 15 17:04:20.401642 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:20.401605    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4c03a4c-ed0f-44d3-86bf-905861139942-maas-api-service-ca\") pod \"a4c03a4c-ed0f-44d3-86bf-905861139942\" (UID: \"a4c03a4c-ed0f-44d3-86bf-905861139942\") "
Jul 15 17:04:20.402000 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:20.401977    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4c03a4c-ed0f-44d3-86bf-905861139942-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a4c03a4c-ed0f-44d3-86bf-905861139942" (UID: "a4c03a4c-ed0f-44d3-86bf-905861139942"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:04:20.503293 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:20.503194    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4c03a4c-ed0f-44d3-86bf-905861139942-maas-api-service-ca\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:04:21.206082 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:21.205992    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-021cd5-x8m44"
Jul 15 17:04:21.206082 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:21.205995    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-021cd5-x8m44" event={"ID":"a4c03a4c-ed0f-44d3-86bf-905861139942","Type":"ContainerDied","Data":"6d165e72ebf926ce33b99c43b48b2ee2e16788adfe0eba59ab1ee4cf7942fec5"}
Jul 15 17:04:21.206311 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:21.206112    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d165e72ebf926ce33b99c43b48b2ee2e16788adfe0eba59ab1ee4cf7942fec5"
Jul 15 17:04:27.675896 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:27.675860    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m"]
Jul 15 17:04:27.676383 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:27.676132    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m" podUID="1b44499b-577e-4e2e-8f40-ebcf3ef5098a" containerName="maas-api" containerID="cri-o://40403f8f9fcfed9919e4bad2a5fce4efecefe89cec1e6b6aacd747c5e8905200" gracePeriod=30
Jul 15 17:04:27.713288 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:27.713234    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:04:27.715178 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:27.715137    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-021cd5-29735580-ddzvx"]
Jul 15 17:04:27.717217 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:27.717189    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-021cd5-29735580-ddzvx"]
Jul 15 17:04:27.921292 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:27.921265    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m"
Jul 15 17:04:27.967135 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:27.967058    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b44499b-577e-4e2e-8f40-ebcf3ef5098a-openshift-service-ca\") pod \"1b44499b-577e-4e2e-8f40-ebcf3ef5098a\" (UID: \"1b44499b-577e-4e2e-8f40-ebcf3ef5098a\") "
Jul 15 17:04:27.967135 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:27.967092    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2w9v\" (UniqueName: \"kubernetes.io/projected/1b44499b-577e-4e2e-8f40-ebcf3ef5098a-kube-api-access-n2w9v\") pod \"1b44499b-577e-4e2e-8f40-ebcf3ef5098a\" (UID: \"1b44499b-577e-4e2e-8f40-ebcf3ef5098a\") "
Jul 15 17:04:27.967135 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:27.967127    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b44499b-577e-4e2e-8f40-ebcf3ef5098a-maas-api-tls\") pod \"1b44499b-577e-4e2e-8f40-ebcf3ef5098a\" (UID: \"1b44499b-577e-4e2e-8f40-ebcf3ef5098a\") "
Jul 15 17:04:27.967424 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:27.967357    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b44499b-577e-4e2e-8f40-ebcf3ef5098a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1b44499b-577e-4e2e-8f40-ebcf3ef5098a" (UID: "1b44499b-577e-4e2e-8f40-ebcf3ef5098a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 17:04:27.969193 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:27.969165    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b44499b-577e-4e2e-8f40-ebcf3ef5098a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1b44499b-577e-4e2e-8f40-ebcf3ef5098a" (UID: "1b44499b-577e-4e2e-8f40-ebcf3ef5098a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 17:04:27.969296 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:27.969206    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b44499b-577e-4e2e-8f40-ebcf3ef5098a-kube-api-access-n2w9v" (OuterVolumeSpecName: "kube-api-access-n2w9v") pod "1b44499b-577e-4e2e-8f40-ebcf3ef5098a" (UID: "1b44499b-577e-4e2e-8f40-ebcf3ef5098a"). InnerVolumeSpecName "kube-api-access-n2w9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 17:04:28.068518 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:28.068484    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b44499b-577e-4e2e-8f40-ebcf3ef5098a-openshift-service-ca\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:04:28.068518 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:28.068515    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2w9v\" (UniqueName: \"kubernetes.io/projected/1b44499b-577e-4e2e-8f40-ebcf3ef5098a-kube-api-access-n2w9v\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:04:28.068518 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:28.068525    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b44499b-577e-4e2e-8f40-ebcf3ef5098a-maas-api-tls\") on node \"ip-10-0-134-132.ec2.internal\" DevicePath \"\""
Jul 15 17:04:28.247851 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:28.247739    2580 generic.go:356] "Generic (PLEG): container finished" podID="1b44499b-577e-4e2e-8f40-ebcf3ef5098a" containerID="40403f8f9fcfed9919e4bad2a5fce4efecefe89cec1e6b6aacd747c5e8905200" exitCode=0
Jul 15 17:04:28.247851 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:28.247799    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m"
Jul 15 17:04:28.247851 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:28.247823    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m" event={"ID":"1b44499b-577e-4e2e-8f40-ebcf3ef5098a","Type":"ContainerDied","Data":"40403f8f9fcfed9919e4bad2a5fce4efecefe89cec1e6b6aacd747c5e8905200"}
Jul 15 17:04:28.248090 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:28.247881    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m" event={"ID":"1b44499b-577e-4e2e-8f40-ebcf3ef5098a","Type":"ContainerDied","Data":"ee05ee1cde9eb13d0047ec3e248c03c18d339d8cebb96f496effeaa7b45d1872"}
Jul 15 17:04:28.248090 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:28.247913    2580 scope.go:117] "RemoveContainer" containerID="40403f8f9fcfed9919e4bad2a5fce4efecefe89cec1e6b6aacd747c5e8905200"
Jul 15 17:04:28.263009 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:28.262974    2580 scope.go:117] "RemoveContainer" containerID="40403f8f9fcfed9919e4bad2a5fce4efecefe89cec1e6b6aacd747c5e8905200"
Jul 15 17:04:28.263302 ip-10-0-134-132 kubenswrapper[2580]: E0715 17:04:28.263257    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40403f8f9fcfed9919e4bad2a5fce4efecefe89cec1e6b6aacd747c5e8905200\": container with ID starting with 40403f8f9fcfed9919e4bad2a5fce4efecefe89cec1e6b6aacd747c5e8905200 not found: ID does not exist" containerID="40403f8f9fcfed9919e4bad2a5fce4efecefe89cec1e6b6aacd747c5e8905200"
Jul 15 17:04:28.263400 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:28.263293    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40403f8f9fcfed9919e4bad2a5fce4efecefe89cec1e6b6aacd747c5e8905200"} err="failed to get container status \"40403f8f9fcfed9919e4bad2a5fce4efecefe89cec1e6b6aacd747c5e8905200\": rpc error: code = NotFound desc = could not find container \"40403f8f9fcfed9919e4bad2a5fce4efecefe89cec1e6b6aacd747c5e8905200\": container with ID starting with 40403f8f9fcfed9919e4bad2a5fce4efecefe89cec1e6b6aacd747c5e8905200 not found: ID does not exist"
Jul 15 17:04:28.274195 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:28.274168    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m"]
Jul 15 17:04:28.278280 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:28.278255    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-021cd5-6787dd4779-tdx6m"]
Jul 15 17:04:28.878493 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:28.878463    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b44499b-577e-4e2e-8f40-ebcf3ef5098a" path="/var/lib/kubelet/pods/1b44499b-577e-4e2e-8f40-ebcf3ef5098a/volumes"
Jul 15 17:04:28.878874 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:28.878732    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72e50f4b-5a83-4c1c-9d32-03b8041cf57c" path="/var/lib/kubelet/pods/72e50f4b-5a83-4c1c-9d32-03b8041cf57c/volumes"
Jul 15 17:04:30.788497 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:30.788417    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-mlqmb"]
Jul 15 17:04:32.235110 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:32.235074    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:04:38.543212 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:38.543169    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:04:38.642961 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:38.642924    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:04:42.959696 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:42.959659    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:04:43.217208 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:43.217094    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:04:50.283707 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:50.283678    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-599cdff4cb-wxbdl_fa739bac-a448-47d5-9af0-8fca1c693851/copy-manifests/0.log"
Jul 15 17:04:50.294498 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:50.294469    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-599cdff4cb-wxbdl_fa739bac-a448-47d5-9af0-8fca1c693851/manager/0.log"
Jul 15 17:04:50.801635 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:50.801605    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735550-s5msg_99028ae3-a363-49d2-b2c3-b4a73e1e378f/cleanup/0.log"
Jul 15 17:04:50.921575 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:50.921546    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735565-4qmlx_7eced033-1cb5-4264-9758-038529a62d22/cleanup/0.log"
Jul 15 17:04:51.039834 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:51.039802    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735580-jx74m_329af7e1-f343-4973-a46a-0ae1495a58b3/cleanup/0.log"
Jul 15 17:04:51.160597 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:51.160527    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-3ac1ded3-b99cz_9ec7c91b-db8b-48ba-acd0-c6feb8ba7ef6/revoke-keys/0.log"
Jul 15 17:04:51.285553 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:51.285529    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-51d2510e-dkbxf_d0fe66db-3795-4b1b-98ec-870f9466e9a9/revoke-keys/0.log"
Jul 15 17:04:51.406537 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:51.406514    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-00e709-gtsgc_92e59ceb-5db3-46c3-a465-316af0c3b0ba/revoke-keys/0.log"
Jul 15 17:04:51.526683 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:51.526600    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-90f61e-2bx6j_ab2c263f-62a3-40c4-b394-6d68d26f6453/revoke-keys/0.log"
Jul 15 17:04:51.650812 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:51.650767    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-77b034d1-scdnv_02ceffc6-50fd-4ef8-96db-45ff4e0cd4e2/revoke-keys/0.log"
Jul 15 17:04:51.777669 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:51.777589    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-5b944f-r48cp_9a5a903b-dcd4-4321-9106-08dec8627cf0/revoke-keys/0.log"
Jul 15 17:04:51.900715 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:51.900689    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-cd9127-lk7c2_ecfc41f1-5028-491f-8d95-cd3a2564b841/revoke-keys/0.log"
Jul 15 17:04:52.024610 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:52.024578    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-439e5009-69jj9_deccb51a-4f6a-4404-a271-6a5456d7fda2/revoke-keys/0.log"
Jul 15 17:04:52.159840 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:52.159816    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-19642093-6jkmk_574a5817-75e6-4912-a9ab-862438434d15/revoke-keys/0.log"
Jul 15 17:04:52.275223 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:52.275194    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-dc2354a8-dxfmz_08a89f85-c4ba-4f8d-b669-651314356715/revoke-keys/0.log"
Jul 15 17:04:52.397505 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:52.397476    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-8269b5-mgjg5_3ca7c91e-c6c1-4c33-b7e3-37c2db6ce496/revoke-keys/0.log"
Jul 15 17:04:52.528309 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:52.528228    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-021cd5-x8m44_a4c03a4c-ed0f-44d3-86bf-905861139942/revoke-keys/0.log"
Jul 15 17:04:54.189769 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:54.189734    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-kcxqf_285b6be6-04b2-472c-bac0-c15063da586f/manager/0.log"
Jul 15 17:04:54.309073 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:54.309040    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-jnvd9_644bc2c5-b077-4139-9ffe-48348c359e91/manager/0.log"
Jul 15 17:04:54.565185 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:54.565152    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-7w6s2_a9ad7292-25ce-4a79-9601-daabe3cc9c7a/registry-server/0.log"
Jul 15 17:04:54.696406 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:54.696378    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-xhw65_34d0da2c-f49c-4747-87d1-d94807f4ce45/manager/0.log"
Jul 15 17:04:54.827893 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:54.827813    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-mlqmb_62bd54a0-d8bf-4314-ab69-849a178c7cf9/limitador/0.log"
Jul 15 17:04:54.958747 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:54.958719    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-zxf6c_5b3f1b15-3376-46d0-8a45-aec700393f71/manager/0.log"
Jul 15 17:04:55.476712 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:55.476682    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-9t5cw_6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f/discovery/0.log"
Jul 15 17:04:55.711858 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:55.711826    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84484457c8-wvxvj_99a21c19-71ef-400d-83a9-b2287f0f7c1f/kube-auth-proxy/0.log"
Jul 15 17:04:56.705071 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:56.705043    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-tnpg4_7017539a-d855-4338-94b7-69efcbecee4c/main/0.log"
Jul 15 17:04:58.031137 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:04:58.031100    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:05:02.023623 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:02.023575    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:05:03.321206 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:03.321167    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:05:03.467631 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:03.467597    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:05:06.149092 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:06.149063    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-c75b9_7549f9c6-5988-4771-ae95-99e674252273/global-pull-secret-syncer/0.log"
Jul 15 17:05:06.289519 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:06.289485    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log"
Jul 15 17:05:06.384430 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:06.384396    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-132.ec2.internal_dc2084eaf308dac38296e58e829ee36f/haproxy/0.log"
Jul 15 17:05:11.948785 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:11.948738    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-kcxqf_285b6be6-04b2-472c-bac0-c15063da586f/manager/0.log"
Jul 15 17:05:11.972393 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:11.972358    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-jnvd9_644bc2c5-b077-4139-9ffe-48348c359e91/manager/0.log"
Jul 15 17:05:12.056893 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:12.056851    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-7w6s2_a9ad7292-25ce-4a79-9601-daabe3cc9c7a/registry-server/0.log"
Jul 15 17:05:12.213458 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:12.213350    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-xhw65_34d0da2c-f49c-4747-87d1-d94807f4ce45/manager/0.log"
Jul 15 17:05:12.240653 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:12.240621    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-mlqmb_62bd54a0-d8bf-4314-ab69-849a178c7cf9/limitador/0.log"
Jul 15 17:05:12.351431 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:12.351397    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-zxf6c_5b3f1b15-3376-46d0-8a45-aec700393f71/manager/0.log"
Jul 15 17:05:14.197138 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:14.197050    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-2z6kg_7ab4d8ae-d58c-4d06-b208-73a036fd4f69/kube-state-metrics/0.log"
Jul 15 17:05:14.214015 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:14.213987    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-2z6kg_7ab4d8ae-d58c-4d06-b208-73a036fd4f69/kube-rbac-proxy-main/0.log"
Jul 15 17:05:14.235385 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:14.235360    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-2z6kg_7ab4d8ae-d58c-4d06-b208-73a036fd4f69/kube-rbac-proxy-self/0.log"
Jul 15 17:05:14.415295 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:14.415266    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7g2q8_2241b2de-66eb-497c-b71c-34b725a04409/node-exporter/0.log"
Jul 15 17:05:14.435835 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:14.435800    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7g2q8_2241b2de-66eb-497c-b71c-34b725a04409/kube-rbac-proxy/0.log"
Jul 15 17:05:14.457020 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:14.456924    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7g2q8_2241b2de-66eb-497c-b71c-34b725a04409/init-textfile/0.log"
Jul 15 17:05:14.660274 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:14.660241    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b65c1c2-a69a-444b-868e-2e8265cff8f3/prometheus/0.log"
Jul 15 17:05:14.681119 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:14.681092    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b65c1c2-a69a-444b-868e-2e8265cff8f3/config-reloader/0.log"
Jul 15 17:05:14.703446 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:14.703418    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b65c1c2-a69a-444b-868e-2e8265cff8f3/thanos-sidecar/0.log"
Jul 15 17:05:14.726696 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:14.726608    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b65c1c2-a69a-444b-868e-2e8265cff8f3/kube-rbac-proxy-web/0.log"
Jul 15 17:05:14.750695 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:14.750654    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b65c1c2-a69a-444b-868e-2e8265cff8f3/kube-rbac-proxy/0.log"
Jul 15 17:05:14.773196 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:14.773175    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b65c1c2-a69a-444b-868e-2e8265cff8f3/kube-rbac-proxy-thanos/0.log"
Jul 15 17:05:14.796942 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:14.796913    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b65c1c2-a69a-444b-868e-2e8265cff8f3/init-config-reloader/0.log"
Jul 15 17:05:16.035046 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.035010    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4vtsf/perf-node-gather-daemonset-cgb5l"]
Jul 15 17:05:16.035550 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.035528    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b44499b-577e-4e2e-8f40-ebcf3ef5098a" containerName="maas-api"
Jul 15 17:05:16.035624 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.035554    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b44499b-577e-4e2e-8f40-ebcf3ef5098a" containerName="maas-api"
Jul 15 17:05:16.035624 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.035593    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4c03a4c-ed0f-44d3-86bf-905861139942" containerName="revoke-keys"
Jul 15 17:05:16.035624 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.035603    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4c03a4c-ed0f-44d3-86bf-905861139942" containerName="revoke-keys"
Jul 15 17:05:16.035791 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.035684    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4c03a4c-ed0f-44d3-86bf-905861139942" containerName="revoke-keys"
Jul 15 17:05:16.035791 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.035698    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b44499b-577e-4e2e-8f40-ebcf3ef5098a" containerName="maas-api"
Jul 15 17:05:16.039260 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.039238    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4vtsf/perf-node-gather-daemonset-cgb5l"
Jul 15 17:05:16.042016 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.041992    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4vtsf\"/\"kube-root-ca.crt\""
Jul 15 17:05:16.042954 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.042936    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4vtsf\"/\"default-dockercfg-cgjtb\""
Jul 15 17:05:16.043011 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.042936    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4vtsf\"/\"openshift-service-ca.crt\""
Jul 15 17:05:16.050376 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.050354    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4vtsf/perf-node-gather-daemonset-cgb5l"]
Jul 15 17:05:16.061483 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.061456    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2xdt\" (UniqueName: \"kubernetes.io/projected/6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94-kube-api-access-g2xdt\") pod \"perf-node-gather-daemonset-cgb5l\" (UID: \"6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94\") " pod="openshift-must-gather-4vtsf/perf-node-gather-daemonset-cgb5l"
Jul 15 17:05:16.061637 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.061516    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94-podres\") pod \"perf-node-gather-daemonset-cgb5l\" (UID: \"6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94\") " pod="openshift-must-gather-4vtsf/perf-node-gather-daemonset-cgb5l"
Jul 15 17:05:16.061637 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.061549    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94-lib-modules\") pod \"perf-node-gather-daemonset-cgb5l\" (UID: \"6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94\") " pod="openshift-must-gather-4vtsf/perf-node-gather-daemonset-cgb5l"
Jul 15 17:05:16.061637 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.061593    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94-proc\") pod \"perf-node-gather-daemonset-cgb5l\" (UID: \"6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94\") " pod="openshift-must-gather-4vtsf/perf-node-gather-daemonset-cgb5l"
Jul 15 17:05:16.061637 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.061635    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94-sys\") pod \"perf-node-gather-daemonset-cgb5l\" (UID: \"6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94\") " pod="openshift-must-gather-4vtsf/perf-node-gather-daemonset-cgb5l"
Jul 15 17:05:16.162692 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.162657    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2xdt\" (UniqueName: \"kubernetes.io/projected/6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94-kube-api-access-g2xdt\") pod \"perf-node-gather-daemonset-cgb5l\" (UID: \"6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94\") " pod="openshift-must-gather-4vtsf/perf-node-gather-daemonset-cgb5l"
Jul 15 17:05:16.162905 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.162711    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94-podres\") pod \"perf-node-gather-daemonset-cgb5l\" (UID: \"6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94\") " pod="openshift-must-gather-4vtsf/perf-node-gather-daemonset-cgb5l"
Jul 15 17:05:16.162905 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.162734    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94-lib-modules\") pod \"perf-node-gather-daemonset-cgb5l\" (UID: \"6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94\") " pod="openshift-must-gather-4vtsf/perf-node-gather-daemonset-cgb5l"
Jul 15 17:05:16.162905 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.162784    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94-proc\") pod \"perf-node-gather-daemonset-cgb5l\" (UID: \"6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94\") " pod="openshift-must-gather-4vtsf/perf-node-gather-daemonset-cgb5l"
Jul 15 17:05:16.162905 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.162813    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94-sys\") pod \"perf-node-gather-daemonset-cgb5l\" (UID: \"6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94\") " pod="openshift-must-gather-4vtsf/perf-node-gather-daemonset-cgb5l"
Jul 15 17:05:16.162905 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.162875    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94-sys\") pod \"perf-node-gather-daemonset-cgb5l\" (UID: \"6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94\") " pod="openshift-must-gather-4vtsf/perf-node-gather-daemonset-cgb5l"
Jul 15 17:05:16.162905 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.162894    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94-proc\") pod \"perf-node-gather-daemonset-cgb5l\" (UID: \"6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94\") " pod="openshift-must-gather-4vtsf/perf-node-gather-daemonset-cgb5l"
Jul 15 17:05:16.163164 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.162911    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94-podres\") pod \"perf-node-gather-daemonset-cgb5l\" (UID: \"6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94\") " pod="openshift-must-gather-4vtsf/perf-node-gather-daemonset-cgb5l"
Jul 15 17:05:16.163164 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.162925    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94-lib-modules\") pod \"perf-node-gather-daemonset-cgb5l\" (UID: \"6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94\") " pod="openshift-must-gather-4vtsf/perf-node-gather-daemonset-cgb5l"
Jul 15 17:05:16.171345 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.171320    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2xdt\" (UniqueName: \"kubernetes.io/projected/6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94-kube-api-access-g2xdt\") pod \"perf-node-gather-daemonset-cgb5l\" (UID: \"6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94\") " pod="openshift-must-gather-4vtsf/perf-node-gather-daemonset-cgb5l"
Jul 15 17:05:16.350659 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.350626    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4vtsf/perf-node-gather-daemonset-cgb5l"
Jul 15 17:05:16.484417 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.484392    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4vtsf/perf-node-gather-daemonset-cgb5l"]
Jul 15 17:05:16.486314 ip-10-0-134-132 kubenswrapper[2580]: W0715 17:05:16.486280    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6b5f1d5f_b9b7_4f6c_93ac_0734e0b11d94.slice/crio-608e8b567e3cb3d37849cf0a7185324bc0efc974d7508f77d78d3212c3ad5e0c WatchSource:0}: Error finding container 608e8b567e3cb3d37849cf0a7185324bc0efc974d7508f77d78d3212c3ad5e0c: Status 404 returned error can't find the container with id 608e8b567e3cb3d37849cf0a7185324bc0efc974d7508f77d78d3212c3ad5e0c
Jul 15 17:05:16.503925 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:16.503881    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4vtsf/perf-node-gather-daemonset-cgb5l" event={"ID":"6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94","Type":"ContainerStarted","Data":"608e8b567e3cb3d37849cf0a7185324bc0efc974d7508f77d78d3212c3ad5e0c"}
Jul 15 17:05:17.482302 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:17.482251    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-2sqh9_44adf4d8-cfe8-4db9-8c4d-ac493a58493b/download-server/0.log"
Jul 15 17:05:17.511240 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:17.511202    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4vtsf/perf-node-gather-daemonset-cgb5l" event={"ID":"6b5f1d5f-b9b7-4f6c-93ac-0734e0b11d94","Type":"ContainerStarted","Data":"4ec6aee52bde49d9f172802ed44ddf6c8a310cb95564f7b06ccce41cca0e1a06"}
Jul 15 17:05:17.511416 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:17.511328    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4vtsf/perf-node-gather-daemonset-cgb5l"
Jul 15 17:05:17.529391 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:17.529342    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4vtsf/perf-node-gather-daemonset-cgb5l" podStartSLOduration=1.5293257310000001 podStartE2EDuration="1.529325731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 17:05:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:05:17.527227736 +0000 UTC m=+2919.174268271" watchObservedRunningTime="2026-07-15 17:05:17.529325731 +0000 UTC m=+2919.176366323"
Jul 15 17:05:19.193026 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:19.192994    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mng27_7ce6e188-b85d-426c-bd0a-9390979d9f6b/dns/0.log"
Jul 15 17:05:19.216838 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:19.216805    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mng27_7ce6e188-b85d-426c-bd0a-9390979d9f6b/kube-rbac-proxy/0.log"
Jul 15 17:05:19.306744 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:19.306712    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hk4xk_52c1f0a0-9858-45d7-bd19-7d705e9c4e42/dns-node-resolver/0.log"
Jul 15 17:05:19.881810 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:19.881775    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-59798f98df-72z7k_7e5e30f2-bd7d-4003-a535-6d2c79e89602/registry/0.log"
Jul 15 17:05:19.932953 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:19.932926    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7tktr_6019a7f5-5848-4017-a06d-c728a14c4f31/node-ca/0.log"
Jul 15 17:05:21.117256 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:21.117221    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-9t5cw_6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f/discovery/0.log"
Jul 15 17:05:21.168089 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:21.168064    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84484457c8-wvxvj_99a21c19-71ef-400d-83a9-b2287f0f7c1f/kube-auth-proxy/0.log"
Jul 15 17:05:22.063455 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:22.063427    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rkbd4_ce92f3bb-14db-4828-af08-055c33a25a1d/serve-healthcheck-canary/0.log"
Jul 15 17:05:22.532567 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:22.532481    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hkgqt_7a38e9b7-99c1-4867-a5c4-3cb9dead76be/kube-rbac-proxy/0.log"
Jul 15 17:05:22.553516 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:22.553487    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hkgqt_7a38e9b7-99c1-4867-a5c4-3cb9dead76be/exporter/0.log"
Jul 15 17:05:22.576318 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:22.576290    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hkgqt_7a38e9b7-99c1-4867-a5c4-3cb9dead76be/extractor/0.log"
Jul 15 17:05:23.527130 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:23.527100    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4vtsf/perf-node-gather-daemonset-cgb5l"
Jul 15 17:05:25.499286 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:25.499199    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-599cdff4cb-wxbdl_fa739bac-a448-47d5-9af0-8fca1c693851/manager/0.log"
Jul 15 17:05:25.520132 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:25.520103    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-599cdff4cb-wxbdl_fa739bac-a448-47d5-9af0-8fca1c693851/copy-manifests/0.log"
Jul 15 17:05:26.184709 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:26.184621    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:05:26.256912 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:26.256863    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gzwh7_79a480c8-f8ad-4d35-aac1-7753e9895ca6/konnectivity-agent/0.log" limit=1024
Jul 15 17:05:26.411667 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:26.411629    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-9t5cw_6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f/discovery/0.log"
Jul 15 17:05:26.436915 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:26.436839    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84484457c8-wvxvj_99a21c19-71ef-400d-83a9-b2287f0f7c1f/kube-auth-proxy/0.log"
Jul 15 17:05:26.850350 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:26.850320    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-57gsv_7156c58f-c0f1-4b88-a130-248479db8f72/openshift-lws-operator/0.log"
Jul 15 17:05:27.956290 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:27.956197    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-9t5cw_6e8b2f22-aa9a-4b0c-93ea-0da4a88a631f/discovery/0.log"
Jul 15 17:05:27.990897 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:27.990868    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84484457c8-wvxvj_99a21c19-71ef-400d-83a9-b2287f0f7c1f/kube-auth-proxy/0.log"
Jul 15 17:05:33.206021 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:33.205988    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4srwh_30df3a5a-23a5-41c5-a430-7dbfe0fedbe3/kube-multus/0.log"
Jul 15 17:05:33.238003 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:33.237974    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6zrmr_d7494c97-f5f3-47af-a6c7-7f939c53bc39/kube-multus-additional-cni-plugins/0.log"
Jul 15 17:05:33.264070 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:33.264042    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6zrmr_d7494c97-f5f3-47af-a6c7-7f939c53bc39/egress-router-binary-copy/0.log"
Jul 15 17:05:33.287714 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:33.287682    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6zrmr_d7494c97-f5f3-47af-a6c7-7f939c53bc39/cni-plugins/0.log"
Jul 15 17:05:33.313064 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:33.313038    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6zrmr_d7494c97-f5f3-47af-a6c7-7f939c53bc39/bond-cni-plugin/0.log"
Jul 15 17:05:33.340713 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:33.340678    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6zrmr_d7494c97-f5f3-47af-a6c7-7f939c53bc39/routeoverride-cni/0.log"
Jul 15 17:05:33.368990 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:33.368962    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6zrmr_d7494c97-f5f3-47af-a6c7-7f939c53bc39/whereabouts-cni-bincopy/0.log"
Jul 15 17:05:33.398233 ip-10-0-134-132 kubenswrapper[2580]: I0715 17:05:33.398200    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6zrmr_d7494c97-f5f3-47af-a6c7-7f939c53bc39/whereabouts-cni/0.log"